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FY 2005 Report on the NSF Merit Review System 



Summary 



The National Science Foundation received nearly 42,000 new proposals for funding between 
October 1, 2004 and September 30, 2005. The Foundation awarded 23 percent of the proposals 
received, making the decisions through the process of merit review. The merit review process 
includes the steps listed below and depicted in the diagram on the following page: 

•The proposal arrives electronically, and NSF staff see that it is placed with the appropriate 
program(s) for review. 

•The program officer (or team of program officers) reviews the proposal and assigns it to at least 
three experts from outside the Foundation (review generally takes place by mail, advisory panel, 
or combination of mail and advisory panel). Reviewers and panelists use two general criteria: 
intellectual merit and broader impacts. The Division leadership (Division Directors, Deputy 
Division Directors, and/or Section Heads) oversees the review process. The program officer or 
team: 

•selects reviewers and panel members, based on program officer’s knowledge, references 
listed in proposal, recent publications in science and engineering journals, presentations at 
professional meetings, reviewer recommendations, bibliographic and citation databases, and 
proposal author’s suggestions. 

•receives the recommendation of the reviewers/panel, based on merit review criteria and 
other factors such as risk, balance of priorities, and budget constraints. 

•The program officer makes a recommendation to award or decline the proposal, taking into 
account external reviews, panel discussion, and other factors such as portfolio balance and 
amount of funding available. 

•A higher-level official (usually a Division Director, Deputy Division Director, or Section Head) 
reviews all program officer recommendations. For award recommendations, a grants officer in 
the Office of Budget, Finance, and Award Management performs an administrative review. 
Recommendations for large awards receive additional review by higher-level organizations such 
as the Director’s Review Board and the National Science Board. 

•The Division leadership performs an annual assessment of the program portfolio. 

•An external Committee of Visitors (scientists, engineers, and educators) assesses each program 
every 3-5 years, examining the integrity and efficiency of merit review processes and the quality 
of results from the programmatic investments. 

•Advisory Committees (scientists, engineers, educators) review Committee of Visitor reports and 
directorate/office responses and provide guidance to the Foundation’s directorates and offices 
regarding the reports and other matters pertaining to past investments and future research and 
education activities. 
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•The NSF-wide Advisory Committee for Government Performance and Results Act (GPRA) 
Performance Assessment, a single committee of external experts convened yearly to assess 
results, evaluates the Foundation’s portfolios and their linkages to strategic outcome goals. The 
Advisory Committee for GPRA Performance Assessment uses Committee of Visitor reports, 
internal and external directorate assessments of particular programs, investigator project reports, 
and directorate/division collections of outstanding accomplishments from awards in order to 
perform the evaluation. 

•An external contractor performs an independent verification and validation of Foundation 
performance measurement. 

•The National Science Board’s Audit and Oversight Committee reviews the findings presented 
by the Advisory Committee for GPRA Performance Assessment. 

The FY 2005 Report on the NSF Merit Review System provides information about the levels of 
proposal and award activity for the fiscal year 2004 (October 1, 2004 - September 30, 2005) and 
the process by which proposals are reviewed and awarded. A brief list of highlights is provided, 
followed by an introduction and information on numbers of proposals and awards, award sizes, 
and principal investigator and awardee institution characteristics. The next section details the 
steps in the merit review process, and the final section outlines government performance issues 
related to merit review and provides information on special types of proposal and grant 
mechanisms. Appendices include more detailed or illustrative material. This report to the Board 
is required by NSB policy, and has been provided annually since 1977. 
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HIGHLIGHTS 

1. NSF took action on 41,722 competitively reviewed proposals, and provided funding to 9,757 of them 
during FY 2005. This resulted in an overall success rate of 23 percent. The number of proposals 
decreased hy 5 percent compared to FY 2004. Since FY 2000, the number of proposals received has 
increased by 41 percent. 

2. The average annualized award amount for research grants in FY 2005 was $143,662, an increase of 3 
percent above the previous year. 

3. For research grants, the number of people supported by NSF — including graduate students, 
postdoctoral associates, principal investigators, and co-principal investigators — has increased by 1 8 
percent between FY 2000 and FY 2005. The number of graduate students supported in FY 2000 was 
15,650. By FY 2005, the number had climbed to 20,442, representing a 31 percent increase. This 
suggests that larger award sizes can help to build capacity. 

4. In FY 2005, 76 percent of all proposals were processed within six months, compared to 77 percent in 
FY 2004. Once again, the agency exceeded its Government Performance and Results Act (GPRA) 
target goal of 70 percent. 

5. Effective October 1, 2002, NSF returned without review proposals that failed to address separately 
both merit review criteria within the Project Summary. In FY 2005, NSF returned a total of 176 
proposals without review due to the failure to address both merit review criteria. In FY 2004, NSF 
returned a total of 236 proposals without review. 

6. NSF made 387 small grants for exploratory research (SGER) awards in EY 2005 for a total of $27 
million, compared to 382 SGER awards made last year for a total of $29 million. The average size of 
the EY 2005 SGER award was about $70,000, compared to $77,000 in EY 2004 and $68,000 in EY 
2003. NSE will initiate an evaluation of SGERs in EY 2006. 

7. In EY 2005, while the number of proposals received dropped overall by 5 percent compared to the 
previous year, the number of proposals from minority Principal Investigators (Pis) decreased by 3 
percent. The success rate for minority Pis was 23 percent, the same as the overall rate. During EY 
2005, the number of proposals received from women Pis decreased 2 percent. The success rate for 
women Pis was 25 percent, two percentage points higher than the overall rate of 23 percent. 

8. The number of program officers has increased by 4 percent (from 385 to 400) between EY 2004 and 
EY 2005, and the number of science assistants has increased by 9 percent (from 32 to 35). NSE 
continues to examine workforce issues through its business analysis. 

9. In EY 2005, the National Science Board evaluated the Eoundation’s merit review process and 
provided recommendations to improve the transparency and effectiveness of the process. During EY 
2005 and EY 2006 NSE has taken steps toward systematic implementation of the recommended 
actions. 

10. A large number of potentially fundable proposals are declined each year. In EY 2005, close to $1.8 
billion of declined proposals were rated as high as the average rating for an NSE award (4.1 on a 5- 
point scale). These declined proposals represent a rich portfolio of unfunded research and education 
opportunities. 
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FY 2005 Report on the NSF Merit Review System 

1. Introduction 

The National Science Foundation Act of 1950 directs the Foundation "to initiate and support 
basic scientific research and programs to strengthen scientific research potential and science 
education programs at all levels."^ NSF achieves its unique mission by making merit-based 
awards to researchers, educators, and students at approximately 1,700 U.S. colleges, universities 
and other institutions. In Fiscal Year (FY) 2005, NSF awards directly involved an estimated 
195,000 people, including senior researchers, post-doctoral associates, teachers, and students 
from kindergarten through graduate school. 

This year NSF made nearly 10,000 new awards from more than 40,000 competitive proposals 
submitted. Over 96 percent of NSF’s awards are selected through its competitive merit review 
process, combining external and internal evaluation. All proposals for research and education 
projects are evaluated using two criteria: the intellectual merit of the proposed activity and its 
broader impacts, such as impacts on teaching and learning. Reviewers also consider how well 
the proposed activity fosters the integration of research and education and broadens opportunities 
to include a diversity of participants, particularly from underrepresented groups. The merit 
review system is at the very heart of NSF's selection of the projects through which its mission is 
achieved. 

This FY 2005 Report on the NSF Merit Review System responds to a National Science Board 
(NSB) policy endorsed in 1977 and amended in 1984, requesting that the NSF Director submit 
an annual report on the NSF proposal review system. The report provides summary information 
about proposal and award activity and the process by which proposals are reviewed and awarded. 
Section 3 of this year's report describes NSF's response to the recommendations of the Board's 
September 2005 report on NSF's merit review processes.^ 

2. Proposals and Awards 

Competitively Reviewed Proposals, Awards and Success Rates 

During FY 2005, NSF took action on 41,722 competitive, merit reviewed research and education 
proposals, as shown in Text Figure 1, page 7. This represents a slight decrease from the 
previous year. 

During FY 2005, NSF made 9,757 awards, slightly fewer awards than in the previous fiscal year. 
This resulted in an overall success rate of 23 percent. As shown in Appendix Table 1, page 31, 
there are differences in the success rates of the various NSF directorates,^ ranging from 17 



* 42 CFR 16 §1862, available at <http://www4.law.comell.edu/uscode/html/uscode42/usc_sec_42_00001862 — 
000-.html>. 

^ Report of the National Science Board on the National Science Foundation's Merit Review System, NSB-05-1 19. 
Available on the web page at < http://www.nsf.gov/nsb/documents/reports.htm>. 

^ The term “directorates” as used in this report, refers to NSF’s seven programmatic directorates and the Office of 
Polar Programs (OPP). The Office of International Science and Engineering (OISE), formerly a division within the 
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percent for Engineering to 65 percent for the newly established Office of Cyberinfrastructure 
(OCI). The variation may be due to factors such as the relative size and nature of the disciplines 
and communities being served. 



Text Figure 1 

NSF Proposal, Award and Success Rate Trends 



Fiscal Year 





2000 


2001 


2002 


2003 


2004 


2005 


Proposals 


29,508 


31,942 


35,165 


40,075 


43,851 


41,722 


Awards 


9,850 


9,925 


10,406 


10,844 


10,380 


9,757 


Success rate 


33% 


31% 


30% 


27% 


24% 


23% 



The slight decline in proposal submissions from FY 2004 may be explained by the transition of 
the Information Technology Research (ITR) cross-disciplinary focus area back into NSF's core 
research and education programs. A decline in proposals submitted to the Small Business 
Innovation Research (SBIR) program was also observed. 

Types of Proposals and Awards 

In general, NSF makes two kinds of competitive grants for the support of research and education: 

Standard grants provide funding in a single fiscal year award to cover all of the 
proposed activities for the full duration (generally 1-5 years) of a project. 

Continuing grants provide funds for an initial period (usually one year) of a multiple 
year project with a statement of intent to continue funding in yearly increments, called 
“continuing grant increments” or CGIs, until completion of the project. 

Of the 9,757 competitive awards made in FY 2005, 5,943, or 61 percent were standard grants, 
and the rest were continuing grants. In addition to the standard and continuing awards, NSF 
awarded 8,307 continuing grant increments (CGIs) based on proposals that had been 
competitively reviewed in earlier years. ^ As shown below in Text Figure 2 , NSF devotes 21 
percent of its total budget to new standard grants and 16 percent to new continuing grants. The 
use of standard grants allows NSF the flexibility to make new awards each year without carrying 
a large burden of continuing grant obligations. 



Directorate for Social, Behavioral and Economic Sciences, is now located within the NSF Director’s Office. 
Similarly, the Office of Cyberinfrastructure, formerly the Division of Shared Cyberinfrastructure in Computer & 
Information Science & Engineering (CISE), is now located in the NSF Director's Office. See NSF Organization 
Chart in Appendix Table 15, page 59. 

While the original award is a competitive action, the CGI is a non-competitive renewal grant. Continued 
incremental funding is based on NSF review of annual project reports and additional oversight mechanisms 
established by specific programs. 
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Text Figure 2 

Percentage of NSF Budget by Type of Award 





2000 


2001 


2002 


2003 


2004 


2005 


Standard Grants 


23% 


25% 


26% 


23% 


23% 


21% 


Continuing Grants 


21% 


19% 


21% 


21% 


17% 


16% 


Continuing Grant Increments 


38% 


38% 


35% 


36% 


39% 


43% 


Centers, Facilities, and Other^ 


18% 


18% 


18% 


20% 


20% 


20% 


100% = $Billion 


$3.92 


$4.46 


$4.77 


$5.37 


$5.66 


$5.49 



People and Institutions 

NSF’s Strategic Plan (FY 2003 - 2008) includes as an objective the promotion of greater 
diversity in the science and engineering workforce through increased participation of 
underrepresented groups and institutions in all NSF programs and activities. NSF is committed 
to increasing the participation in all NSF activities of researchers, educators and students from 
groups currently underrepresented in the science and engineering enterprise. Success rates over 
the last five fiscal years for all Principal Investigators (Pis), female and minority PIs^, and prior 
and new Pis’ are shown in Text Figure 3 below. Proposals, awards and success rates by PI 
characteristics are presented in Appendix Table 2 on page 32. 

Text Figure 3 

Success rate by Fiscal Year and PI Characteristic 





2000 


2001 


2002 


2003 


2004 


2005 


All 


33% 


31% 


30% 


27% 


24% 


23% 


Female 


35% 


32% 


30% 


28% 


25% 


25% 


Male 


33% 


31% 


30% 


27% 


24% 


23% 


Minority 


32% 


30% 


29% 


27% 


23% 


23% 


New 


25% 


24% 


22% 


19% 


17% 


17% 


Prior 


40% 


36% 


35% 


33% 


29% 


28% 



During FY 2005, the number of proposals submitted to NSF declined slightly from FY 2004 but 
remained about the same or greater than the number submitted in FY 2003. This was true for all 
categories of Pis listed above in Text Figure 3. In FY 2005, the number of proposals received 
dropped overall by 5 percent. The number of proposals from minority Principal Investigators 
(Pis) decreased by 3 percent. The success rate for minority Pis was 23 percent, the same as the 



^ “Other” includes Organizational Excellence activities. 

® Minority includes American Indian or Alaskan Native, Black, Hispanic, and Pacific Islander and excludes Asian 
and White, not of Hispanic Origin. Please note that the data on underrepresented groups are derived from 
information that the principal investigators submit on a voluntary basis. About 90 percent of principal investigators 
supply this information. 

^ A proposal is counted in the New PI category if the PI did not have an NSF award in the current or prior years. 
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overall rate. During FY 2005, the number of proposals received from women Pis decreased 2 
percent. The success rate for women Pis was 25 percent, two percentage points higher than the 
overall rate of 23 percent. Details can be seen in Appendix Table 2, on page 32. In addition. 
Appendix Table 3, page 33, provides a breakdown of success rates by the race/ethnicity of the 
minority Principal Investigators. 

The major gap in success rates continues to be between new Pis and prior Pis (17 percent and 28 
percent, respectively, in FY 2005). There are a number of possible reasons for this; for example, 
prior Pis are more likely to have established research agendas and are thus able to cite the results 
of previously funded projects in their subsequent proposals. In the case of new Pis who have 
conducted research, but are approaching NSF as a funding source for the first time, it may take 
more than one proposal submission to experience success. 

For research grants, the number of people supported by NSF — graduate students, postdoctoral 
associates, principal investigators, and co-principal investigators - has increased by 18 percent 
between FY 2000 and FY 2005. Text Figure 4, below, shows that while the number of NSF 
research grants has decreased from 6,501 in FY 2000 to 6,231 in FY 2005, the number of people 
supported has increased. The increase in the number of graduate students supported has been 
most striking, from 15,650 in FY 2000 to 20,442 in FY 2005, or a 31 percent increase. This 
increase has taken place in the context of increasing award size and suggests that larger award 
sizes can help to build capacity (see below, page 13). 

Text Figure 4 

Competitive Research Awards and People Supported, FY 2000 - 2005 



Year 


2000 


2001 


2002 


2003 


2004 


2005 


% Change, 
2000 - 2005 


Competitive Research Awards 


6,501 


6,221 


6,720 


6,851 


6,510 


6,231 


-4.15% 




Pis Supported 


15,538 


15,827 


16,322 


16,592 


17,013 


16,954 


9.11% 


Pis and Co-PIs Supported 


21,041 


21,739 


22,949 


23,649 


24,612 


24,572 


16.78% 




Postdocs Supported 


3,743 


4,367 


4,320 


4,629 


4,399 


4,068 


8.68% 


Graduate Students Supported 


15,650 


18,717 


19,303 


20,384 


21,105 


20,442 


30.62% 



Note: Competitive Research Awards reflect the new awards made in a given year and do not include 
continuing grant increments. 

Personnel counts reflect all personnel supported in the year, on both competitive research awards and 
continuing increments made on awards reviewed. 

Post-doc and graduate student counts are from the personnel counts reported on research award budgets. 
Data from NSF's Enterprise Information System, as of 7 January 2006. 
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In addition to tracking the success rates of individuals, NSF also looks at success rates for 
academic institutions. For FY 2005, the success rate for research-intensive Ph.D. institutions, 
defined as the top 100 Ph.D. -granting institutions ranked according to the amount of FY 2005 
funding received from NSF, was 25 percent. In comparison, the rate for non-research intensive 
Ph.D. institutions in FY 2005 (i.e., the Ph.D. -granting institutions that are not in the top 100 
NSF-funded category) was 17 percent. Two- and four-year institutions experienced success rates 
of 22 percent and 24 percent, respectively for FY 2005. For minority- serving institutions, the FY 
2005 success rate was 18 percent, down from 20 percent last year. 

In the past year, NSF made a number of outreach presentations to diverse institutions across the 
country in an effort to increase awareness of the NSF merit review process and to encourage the 
submission of proposals submitted by scientists and engineers from underrepresented groups. 
Outreach efforts included workshops for tribal colleges and those institutions eligible for support 

o 

through the Experimental Program to Stimulate Research (EPSCoR). 

Distribution of Budget and Awards by Sector/Institution 

Through its Budget Internet Information System,^ NSE tracks the distribution of dollars by type 
of organization: academic, non-profit, for-profit, and federal. In EY 2005 NSE awarded 76 
percent of its budget to academic institutions, 15 percent to non-profit and other organizations, 7 
percent to for-profit businesses, and 2 percent to Eederal agencies and laboratories. This overall 
distribution of funds by type of organization has remained fairly constant over the past five 
years. 

With regard to academic institutions, NSE classifies them according to the proportion of NSE 
funding they receive. As seen in Text Figure 5, next page, the percentages of NSE awards made 
to the “top funded” (i.e., the institutions receiving the largest proportion of NSE funding) ten, top 
funded fifty, and top funded one hundred academic institutions have varied little over the past 
four years. In EY 2005, the top 10 funded institutions received 17 percent of NSE awards while 
23 percent of NSE awards are made to institutions that are not in the top 100 funded schools. By 
far the largest proportion of dollars went to the top 100 schools (77 percent in EY 2005). NSE 
has as a performance goal for EY 2007 to increase or maintain the percentage of proposals 
received from academic institutions not in the top 100 of NSE funding recipients in several 
investment categories, including fundamental science and engineering.^*^ 



* A description of outreach events, both past and planned, is available on the NSF web page at 
<http://www.nsf.gov/events/>. 

^ The Budget Internet Information System is available on the web at < http://dellweb.bfa.nsf.gov/>. 

*** See NSF's FY 2007 Budget Request to Congress, 6 February 2006, "Performance Information, "available on the 
web at <http://www.nsf.gOv/about/budget/fy2007/toc.jsp>. 
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Text Figure 5 

Percent of Awards to Top Funded Academic Institutions 
Fiscal Year 2002 - 2005 
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Award Amounts and Duration 

The average annualized award amount for research grants^ ^ in FY 2005 was $143,669, an 
increase of 3 percent from the previous year. The median award^^ was $103,965, a 2 percent 
increase over the previous year.*^ Text Figure 6, next page, displays average and median NSF 
award amounts from FY 1998 to FY 2005. Data by NSF directorate for the last five years are 
presented in Appendix Table 4, page 33. 

Adequate award size is important both for attracting high-quality proposals and for ensuring that 
proposed work can be accomplished as planned. Larger awards permit the participation of more 
students and allow scientists and engineers to devote a greater portion of their time to actual 
research rather than writing and reviewing proposals. 



** Research Grants is a subset of total NSF awards associated primarily with individual investigator and group 
research projects. These do not include education and training grants, which are primarily multi-institution and of a 
much larger average size. 

The difference between the median and average award amounts reflects the effect of numerous small awards on 
the median, and a few large awards on the average award amount. 

Beginning in FY 2003 collaborative proposals submitted as individual proposals from the collaborating 
institutions were counted as a single proposal as NSF treats them as a single proposal for review and award/decline 
decisions. If collaborative proposals were counted individually in FY 2003, the average award size would have been 
$121,380. 
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Text Figure 6 
Award Amounts 

Competitively Reviewed Research Awards 
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Longer award terms are important in increasing the effectiveness of principal investigators and 
graduate students. Graduate students are able to have more time to do their thesis work. NSF’s 
FY 2005 GPRA goal was to achieve an average award duration of 3.0 years for research grants. 
The actual result was 2.96 years, thus NSF was not successful for this goal. Program directors 
must balance competing requirements, such as increasing award size, increasing duration of 
awards, and/or making more awards. NSF will continue to give careful attention to award size 
and duration in the context of recent declines in success rates. 

Proposal Processing Efficiency - Time to Decision (Proposal Dwell Time) 

It is important for applicants to receive a timely funding decision. NSF’s FY 2005 GPRA 
performance goal was, for at least 70 percent of proposals, to inform applicants whether their 
proposals have been declined or recommended for funding within six months of receipt. As 
indicated in Text Figure 7, below, NSF surpassed this goal. In FY 2005, 76 percent of all 
proposals were processed within six months, slightly less than in FY 2004. The achievement of 
this goal is particularly significant because of the trend toward major increases in the number of 
proposals submitted, thus adding to the merit review workload of program staff. 

Text Figure 7 
Proposal Dwell Time 

Percentage of Proposals Processed Within 6 Months 



Fiscal Year 


2001 


2002 


2003 


2004 


2005 


Percentage 


63% 


74% 


77% 


77% 


76% 



3. The Proposal Review Process 

The NSF proposal process starts with electronic receipt of the proposal, which is then forwarded 
electronically to the appropriate NSF program for review. All proposals are reviewed by a 
scientist, engineer, or educator serving as an NSF program officer, and usually by three or more 
experts from outside NSF in the particular fields represented in the proposal. Program officers at 
NSF follow the merit review process guidelines found in NSF Manual #10, The Proposal and 
Award Manual, Chapter V, available on the internal NSF web page. For example, the program 
officer exercises care to assure that the external reviewers have no conflicts of interest. The 
Proposal and Award Manual also requires a minimum of three external reviews for most 
proposals. 

Proposers may suggest names of persons they believe are especially well qualified to review the 
proposal, along with persons who they believe should not review the proposal. These 
suggestions may serve as an additional resource in the reviewer selection process, at the program 
officer’s discretion. Program officers also rely on their knowledge of what is being done by 
whom in their research and education area, the references listed in the proposal, recent 
publications and professional meetings, bibliographic databases, and recommendations from 
other reviewers. Program officers may obtain comments from assembled review panels or from 
site visit teams before recommending final action on proposals. 
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Senior NSF staff at the division or section level further review the program officer’s 
recommendations for awards and declinations. When a decision has been made, verbatim copies 
of reviews, excluding the names of the reviewers, and summaries of review panel deliberations, 
if any, are provided to the proposal author. 

Review Processes Used at NSF 

NSF’s proposal review system relies on extensive use of knowledgeable experts from outside the 
Foundation. Expert judgments of which proposals best address the merit review criteria 
established by the National Science Board inform Foundation staff and influence funding 
recommendations. NSF programs obtain external peer review by three principal methods: (1) 
“mail-only,” (2) “panel-only,” and (3) “mail-plus -panel” review. In addition, site visits by NSF 
staff and external members of the community are often used to review proposals for facilities and 
centers. NSF program officers are given discretion in the specific use of review methods, subject 
to higher-level review. 

In the “mail-only” review method, reviewers are sent proposals and asked to submit written 
comments to NSF through FastLane, NSF’s web-based system for electronic proposal 
submission and review. These mail reviews are then used by the NSF program officer in his or 
her decision to recommend an award or declination. 

“Panel-only” review refers to the process of soliciting reviews only from those who meet in a 
panel review setting to discuss their reviews and provide advice directly to the program officer. 
Most programs that use this process provide proposals to panelists and receive their reviews 
before the panel meeting. The program officer uses this panel advice to decide to recommend an 
award or declination. 

Many proposals submitted to NSF are reviewed using some combination of these two processes 
(“mail-plus-panel” review). Those programs that employ the mail-plus-panel review process 
have developed several different configurations, such as: 

• A reviewer is asked to submit a written mail review and also serve as a panelist; and 

• A reviewer is asked to participate only as a panelist, with responsibility only for 
reviewing and discussing mail reviews written by others and providing verbal and/or 
written advice to the program officer. 

The use of various review methods has changed markedly over time, as shown below in Text 
Figure 8, next page, and the corresponding Appendix Table 5 (page 34). Between 1996 and 

2004, the percentage of NSF proposals reviewed by panel-only increased from 41 to 56 percent 
of all proposal. In FY 2005, the percentage of proposals reviewed by panel-only dipped slightly 
from 56 to 54. From 1996 through 2005, there has been a steady decline in the use of mail-only 
review from 26 to 9 percent. The use of mail-plus-panel review increased to 33 percent in FY 

2005 . *^ 



During this period of about 10 years, between three and six percent of the proposals were not externally reviewed; 
these include proposals for conferences or symposia, small grants for exploratory research, and other special types 
of proposals that are subject to internal but not external review. 
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There are a number of reasons for the trend away from mail-review only. Panels allow reviewers 
to discuss and compare proposals. Panels tend to be used for programs that rely on concrete 
deadlines as opposed to target dates. The panel review process has advantages in the evaluation 
of multidisciplinary proposals or interdisciplinary proposals in new or developing research areas 
because, unlike mail-only review, viewpoints representing several disciplines can be openly 
discussed and integrated. In a similar fashion, the panel review discussion facilitates 
consideration of both merit review criteria. Finally, the panel review process usually requires 
fewer individual reviewers per proposal than the mail-only process. A panel of 25 reviewers 
could possibly review 200 proposals, while it may require several hundred requests for mail 
reviewers to review the same proposals. Also, using panels in the review process tends to reduce 
proposal processing time (time-to-decision), compared to mail-only reviews. For example, in FY 
2005, 79 percent of all proposals reviewed by panel-only were processed within six months, 
compared to 73 percent for mail-plus-panel and 59 percent for mail-only. 

Text Figure 8 

FY 1996 - 2005 Trend, NSF Review Method 
(Percentage of Proposals) 




■*— mail only reviews mail+panel reviews — not reviewed — panel only reviews 



Mail review often takes more time because additional reviews must be requested when some of 
the reviewers in the first set decline to review the proposal. The chief advantages of mail review 
are: (1) the expertise of the reviewers can be more precisely matched to the proposal, and (2) it 
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is less expensive (there are no travel costs). The mail-plus-panel review process is used 
frequently because it combines the in-depth expertise of mail review with the more comparative 
analysis of panel review. 

Some programs are continuing to experiment with “virtual panels”. In virtual panels, panelists 
participate from their offices or homes and interact electronically using NSF’s Interactive Panel 
System (IPS), accompanied by a teleconference. Around 95 percent of panels, whether they 
assemble at NSF, offsite at a common location, or virtually, are using IPS. A part of FastLane, 
IPS permits the viewing of proposals, reviews, basic panel discussions, collaboration on panel 
summaries, and approval of the draft panel summary through the web. Some programs are 
making use of NSF’s videoconferencing facilities to enhance the participation of panelists whose 
schedules do not permit them to be physically present at the time of the panel. 

Videoconferencing is also employed in award management and oversight for large center-type 
projects. NSF is continuing its efforts to improve web-based and electronic means of 
communication to contribute to the quality of the merit review and award oversight processes. 

Directorate-level data on the use of different review processes during FY 2005 are presented in 
Appendix Table 6, page 34. NSF Directorates vary in their use of proposal review methods. 
Mail-plus-panel review was the predominant review process used in the Office of Polar 
Programs and the Directorates of Biological Sciences, Geosciences, and Social, Behavioral and 
Economic Sciences, while panel-only review was the predominant method in the Office of 
Cyberinfrastructure and the Directorates of Computer & Information Science & Engineering, 
Education and Human Resources, Engineering, and Mathematical & Physical Sciences. Mail- 
only review was the most common mode of review in the Office of International Science and 
Engineering. 

Reviews and Reviewers 

NSE policy, as stated in The Proposal and Award Manual requires at least three external reviews 
for each award or decline recommendation on a proposal, unless the requirement has been 
waived. The total numbers of reviews and the average numbers of reviews per proposal 
obtained by the three different review methods are presented in Text Figure 9. As expected, the 
mail-plus-panel method had the highest number of reviews per proposal, averaging nearly eight, 
while the mail-only method averaged around four. 

Directorate-level data for EY 2005 are presented in Appendix Table 7, page 35. The variation 
among directorates in the number of reviews per proposal reflects both their preferences for the 
different review methods, and differences in the way directorates count reviewers in the panel 
review process. 



See Section V-3 of The Proposal and Award Manual. Exceptions include proposals for Small Grants for 
Exploratory Research (SGER) and workshop and symposia proposals. Eor workshop and symposia proposals, 
however, the program officer may obtain external reviews whenever he or she deems that such review is 
appropriate. 
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Text Figure 9 

Reviews per Proposal, FY 2005 
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NSF maintains a central electronic database of more than 300,000 reviewers. Program officers 
identify potential reviewers using a variety of sources including their own knowledge of the 
discipline, applicant suggestions, references attached to proposals, published papers, scientific 
citation indexes and other similar databases, and input from mail reviewers, panelists, and 
visiting scientists. During FY 2005, approximately 50,000 reviewers were sent one or more 
proposals for mail review. In all, approximately 41,000 individuals served on panels, were sent a 
proposal for mail review, or served in both functions. About 14,000 of these reviewers had never 
reviewed an NSF proposal before. The reviewers came from all 50 states in addition to the 
District of Columbia, Puerto Rico, Virgin Islands, and other U.S. jurisdictions. More than 5,000 
reviewers came from outside of the United States. Moreover, reviewers came from a range of 
institutions, including two-year and four-year colleges and universities. Master’s level and 
Ph.D. -granting universities, industry, and government. FY 2005 data are available on numbers 
of reviewers from each state, territory, and country as well as by type of institution. 

In FY 2001, NSF developed systems and policies to request demographic data electronically 
from all reviewers to determine the participation of underrepresented groups in the NSF reviewer 
pool. The goal was to establish a baseline for participation of underrepresented groups in NSF 
proposal review activities. In FY 2005, out of a total of 40,992 distinct reviewers who returned 
reviews, 8,980 - about 22 percent — provided demographic information. Out of the 8,980 who 
provided information, 3,180 (35%) indicated they were members of an underrepresented group 
(i.e., minority or women; see Note 6, page 8, for a definition of minority). Provision of 
demographic data is voluntary and, given the low response rate, there is not enough information 
to establish a baseline. During FY 2004, NSF altered the FastLane reviewer module to make it 
more convenient for reviewers to provide demographic information and, as a result, NSF has 
seen a slight increase in the proportion of reviewers providing information after the FastLane 
change. In FY 2005 22 percent provided information in comparison to 17 percent in FY 2004. 

NSF continually updates its Library resources, including databases, web pages, and directories, 
and conducts frequent tutorials on finding reviewers. Other activities include the collection and 
sharing of potential reviewer data from associations serving groups that are underrepresented in 
science and engineering, and encouraging participation of members of underrepresented groups 
in NSF workshops and conferences. Some NSF divisions actively solicit new reviewers through 
their web pages and their outreach activities. 
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Participation in the peer review process is voluntary. Panelists are reimbursed for expenses; mail 
reviewers receive no financial compensation. In FY 2005, 60 percent of requests for mail 
reviews elicited positive responses, slightly up from 59 percent in FY 2004 and 58 percent in FY 
2003. 

Merit Review Criteria 

In FY 1998 the National Science Board approved the use of the two current NSF merit review 
criteria now in effect: 

What is the intellectual merit of the proposed activity? How important is the 
proposed activity to advancing knowledge and understanding within its own field or 
across different fields? How well qualified is the proposer (individual or team) to 
conduct the project? (If appropriate, the reviewer will comment on the quality of prior 
work.) To what extent does the proposed activity suggest and explore creative and 
original concepts? How well conceived and organized is the proposed activity? Is there 
sufficient access to resources? 

What are the broader impacts of the proposed activity? How well does the activity 
advance discovery and understanding while promoting teaching, training, and learning? 
How well does the proposed activity broaden the participation of underrepresented 
groups (e.g., gender, ethnicity, disability, geographic, etc.)? To what extent will it 
enhance the infrastructure for research and education, such as facilities, instrumentation, 
networks and partnerships? Will the results be disseminated broadly to enhance scientific 
and technological understanding? What may be the benefits of the proposed activity to 
society? 

In FY 1999 NSF established annual Government Performance and Results Act (GPRA) 
performance goals to increase reviewer and program officer attention to both merit review 
criteria. Currently NSF Committees of Visitors and NSF Staff provide an annual evaluation of 
the Foundation’s use of the merit review criteria. In the National Science Board discussions, 
members expressed concern that the broader impacts criterion was not being fully integrated into 
the review process, and that principal investigators and reviewers are unsure how it should be 
addressed. They agreed that efforts to ensure that both criteria are addressed in proposals and 
reviews should be continued and they asked staff to periodically report on these efforts. 

Since then, the Foundation has completed the following actions to raise awareness of the 
importance and use of the merit review criteria: 

• Provided a set of examples of activities that address the broader impacts criterion and 
made the examples available to proposers via a link embedded in the Grant Proposal 
Guide (http://www.nsf.gov/pubs/gpg/broaderimpacts.pdf). In addition, the examples are 
available to proposers and reviewers via FastLane. 

• Revised the FastLane Proposal Preparation Guidelines and the standard language in the 
Program Information Management System (PIMS) that instructs proposers that they must 
clearly address broader impacts in the project summaries of their proposals. 
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• Provided guidance to proposers in the Grant Proposal Guide that Principal Investigators 
must address both merit review criteria in separate statements within the one page Project 
Summary. The Grant Proposal Guide also reiterates that broader impacts resulting from 
the proposed project must be addressed in the project description and described as an 
integral part of the narrative. Effective October 1, 2002, NSF returned without review 
proposals that failed to separately address both merit review criteria within the project 
summary. For FY 2005, 176 proposals were returned with out review due to the failure 
to address the merit review criteria in the summary; the number of returned proposals for 
the previous fiscal year was 236. 

• Revised guidance in the Proposal and Award Manual to require program officers to 
comment on both the intellectual merit and the broader impacts of the proposed activity 
as part of the review analysis of the proposal. 

• Updated NSF's reviewer forms to provide the capability for reviewers to comment 
separately on both criteria in the review of a proposal. 

• Examined reviewer use of the broader impacts criterion and concluded that 
approximately 90 percent of reviews addressed both intellectual merit and broader 
impacts in the last three fiscal years. 

Reviewer Proposal Ratings 

The NSF merit review system emphasizes reviewer narratives in addition to numerical ratings. 
The written comments provided by reviewers, the summary of panel discussions, and the expert 
judgments of program officers are important components of the merit review system. Summary 
ratings are another indicator of reviewer judgment. The distribution of average summary ratings 
of reviews for awarded and declined proposals is provided in Text Figure 10, next page. 
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These data indicate considerable overlap among the average reviewer ratings of successful and 
unsuccessful proposals, most notably in the range of “very good” average ratings/^ Appendix 
Tables 8-10, pages 36-38, indicate that this overlap among the average reviewer ratings is 
present and similar in degree for each of the three proposal review methods used by NSF (panel- 
only, mail-only, and mail plus panel). 

NSF Program Officer Recommendations 

As noted above, the narrative comments and summary ratings provided by external reviewers are 
essential inputs that inform the judgment of the program officers who formulate award and 
decline recommendations to NSF’s senior management. 



The corresponding numerical ratings, on a five-point scale, are as follows: Excellent (4.5 - 5.0); Very Good - 
Excellent (4.0 - <4.5); Good - Very Good (3.0 - <4.0); Eair - Good (2.0 - <3.0); and Poor - Fair (<2.0). Proposals 
with “No Score” include small grants for exploratory research and workshop/symposia proposals that do not require 
external review. 
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NSF program officers produce and manage a portfolio of awards. They must balance the 
portfolio among various considerations and objectives. In addition to information contained in 
the external proposal reviews, NSF program officers consider issues such as: 

• Potential impact on human resources and infrastructure; 

• Different approaches to significant research questions; 

• Capacity building in a new and promising research area; 

• Support for high-risk proposals with potential for transformative advances in a field; 

• NSF core strategies, such as the integration of research and education; 

• Achievement of special program objectives and initiatives; 

• Other available funding sources; and 

• Geographic distribution. 

Program Officer Characteristics and Workload 

The number of program officers increased from 385 in FY 2004 to 400 in FY 2005, a 4 percent 
increase. The characteristics of NSF program officers are presented in Text Figure 11. 



Text Figure 11 

Distribution of NSF Program Officers by Characteristics 
As of October 1, 2005 
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Intergovernmental Personnel Act (IP A) 
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Source: NSF Division of Human Resource Management 
Notes: VSEE: Individual employed as a Visiting Scientist, Engineer, or 
Educator (formerly termed “Rotator”). IPA: Individual employed under the 
Intergovernmental Personnel Act. 
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Program Officers can be permanent NSF employees or non-permanent employees (includes 
Visiting Scientist, Engineer, or Educator; Temporary; and Intergovernmental Personnel Act 
categories). About half of the program officers fall into the non-permanent category. Some non- 
permanent program officers are “on loan” as visiting scientists, engineers, and educators 
(VSEEs) for up to three years from their host institutions. Others are employed through grants to 
the home institutions under the terms of the Intergovernmental Personnel Act (IPA). Non- 
permanent employees provide NSE with new ideas and fresh science and engineering 
perspectives. They bring knowledge of the most recent disciplinary and interdisciplinary 
developments to enhance NSE’s responsiveness and agility. Whether they are hired as temporary 
or permanent, incoming NSE program officers receive training in the merit review process. 

While NSE was able to increase the number of program officers in EY 2005, workload concerns 
are still present and frequently highlighted by NSE's external review committees, the Committees 
of Visitors. NSE developed an overall human capital management plan and is taking steps to 
address the program officer workload issue through, for example, the addition of Science 
Assistant positions. NSE had 35 Science Assistant positions in EY 2005, compared to 32 
positions in EY 2004, a 9 percent increase in these positions. 

NSE has revitalized its professional development opportunities for program staff, offering in- 
house courses in project management, leadership, and communication through the NSE 
Academy. In addition, the Office of Integrative Activities is holding focus groups and forums for 
program staff to share effective practices for merit review, given workload concerns. 

Assuring Objectivity in the Merit Review Process 

NSE program officers check all proposals for potential conflict of interest and select expert 
outside reviewers with no apparent potential conflicts. All reviewers are provided guidance and 
instructed to declare potential conflicts. All program officers receive conflict-of-interest training 
annually. 

Each program officer’s recommendation to award or decline a proposal is subject to a 
programmatic review by a higher level reviewing official (usually the Division Director), and an 
administrative review by a grants officer in the Office of Budget, Einance, and Award 
Management (BEA). The Director’s Review Board (DRB) reviews all award recommendations 
with an average annual award amount of 2.5 percent or more of a Division’s annual budget. The 
National Science Board reviews and approves all recommended awards where the average 
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annual award amount is 1 percent or more of the awarding Directorate's annual budget. 

Every applicant whose proposal undergoes merit review receives a letter stating the results, a 
panel summary explaining the rationale for the decision (if panel review was used), along with 
an anonymous verbatim copy of each review that was considered in the review process. Some 
NSE programs also send "context statements" that explain the broader context under which any 
given proposal was reviewed (e.g., the number of proposals received and the number awarded 
and declined). There is an increasing interest among program staff in the use of context 
statements and other communications to proposers, particularly in the case of difficult 
award/decline recommendations (e.g., proposals with "very good" average ratings). 



Other items requiring NSB prior approval are new programs and major construction projects that meet certain 
specifications. In FY 2005, the Board reviewed and approved nine recommended awards. 
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If, after receiving the reviews and other documentation of the decision, an unsuccessful proposer 
would like additional information, he or she may ask the program officer for further clarification. 
If, after considering the additional information, the applicant is not satisfied that the proposal was 
fairly handled and reasonably reviewed, he or she may request formal reconsideration. In 
response to concerns from the National Science Board and the Office of Inspector General, NSF 
implemented a uniform policy to inform all declined proposers of the reconsideration process 
beginning in FY 2006. A reconsideration request can be based on the applicant’s perception of 
procedural errors or on disagreements over the substantive issues dealt with by reviewers. If the 
AD upholds the original action, the applicant’s institution may request a second reconsideration 
from the Foundation’s Deputy Director. 

NSF declines approximately 30,000 proposals a year but receives only 30-50 requests for formal 
reconsideration. The number of requests for formal reconsideration and resulting decisions at 
both the AD and 0/D levels from FY 2001 through FY 2005 are displayed in Appendix Table 
11 , page 39. NSF had 35 reconsideration requests in FY 2005 and all were upheld. 

NSF's merit review process is evaluated regularly by external Committees of Visitors in addition 
to the Advisory Committee for GPRA Performance Assessment. In FY 2005, the National 
Science Board also evaluated the merit review process, concluding that the NSF merit review 
process is a fair and effective way to review the more than 40,000 proposals the Foundation 
receives annually in a wide variety of subject areas. The Board provided several 
recommendations for NSF to improve the transparency and effectiveness of the NSF merit 
review process, while preserving the ability of the program officers to identify the most 
innovative proposals and effectively diversify and balance NSF's research and education 
portfolio.*^ 

In response to the Board's recommendations, NSF has undertaken an agency-wide effort to 
address quality of reviews, transparency of the award/decline decision, and support of 
transformative research. To date the following actions have been taken: 

• The FY 2007 NSF Performance Plan includes the operation of a credible, efficient merit 
review system as a strategic goal. 

• A merit review performance indicator has been added to the Senior Executive Service 
(SES) annual personal performance plans. 

• Standards have been developed for the Major Research Instrumentation Program and are 
being tested as possible agency-wide standards for the merit review process. 

• Sessions have been conducted with senior staff of all NSE Directorates and Offices to 
raise issues regarding merit review process. All directorates currently have underway 
activities that address the transparency and effectiveness of the merit review process. 

• A focus of the Annual Division Director retreat was the merit review process and 
mechanisms to address quality of review, transparency of award/decline decision, and 
support of transformative research. 



** Please note that certain types of proposals are not eligible for reconsideration. See NSF's Grant Policy Manual, 
Chapter 10, available on the NSF web page at < http://www.nsf.gov/publications/pub_summ.jsp?ods_key=gpm>. 
See Note 2, page 6. 



FY 2005 Report on the NSF Merit Review System 03/06 




25 



• Sessions have been conducted with NSF Administrative Officers (AOs) regarding their 
role and responsibilities in help ensure the quality of documentation of the merit review. 
AOs have supervisory responsibilities for administrative staff and oversee general 
operations. 

• An external web page is being designed to inform the research and education community 
of the NSF review process. 

• An internal NSF web page is being designed to provide merit review process information 
to NSF staff. The website will include the standards expected, effective practices, and 
examples of reviews, panel summaries. Program officer analyses, and program officer 
communications to principal investigators. 

• An enhancement of training sessions for NSF staff is planned. 



4. Other Issues Related to Merit Review 

Budgetary Considerations 

A large number of potentially fundable proposals are declined each year. Text Figure 12, next 
page, indicates that in FY 2005, close to $1.8 billion was requested for declined proposals that 
had received ratings at least as high as the average rating (4.1) for an awarded proposal. These 
declined proposals represent a rich portfolio of unfunded opportunities - fertile ground for 
learning and discovery that lies fallow. There may be a large number of proposals in the declined 
Good to Very Good range of proposals that, if supported, could produce substantial research and 
education benefits. 
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Cumulative Requested Amounts for Declined Proposals 
by Average Reviewer Score for FY 2005 
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Performance Evaluation 



The NSF Strategic Plan for FY 2003-2008 established the goal of Organizational Excellence, and 

the goal was first evaluated in FY 2004. Two external advisory committees led the evaluation: 

the Advisory Committee for Business and Operations (AC/B&O) and the Advisory Committee 

for GPRA Performance Assessment (AC/GPA). The operation of a credible, efficient merit 
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review system has been an important objective within the Organizational Excellence goal. 
Performance evaluation, with respect to the operation of the merit review system, is currently 
supported with information from the following activities: 

• Proposer and Grantee Information/Merit Review. All proposers and grantees provide 
results from previous NSE support, information about existing facilities and equipment 
available to conduct the proposed research, biographical information on the primary 
investigators, other sources of support, and certifications. Such information is required at the 
time of proposal submission, at the time of an award, and in annual and final project reports. 

It is reviewed by NSE staff, used during merit review and included in the package of 
information available to external committees conducting performance assessment. 

• Program Evaluation by Committees of Visitors (COVs). To ensure the highest quality in 
processing and recommending proposals for awards, NSE convenes external groups of 
experts, called Committees of Visitors (COVs), to review each program approximately every 
three to five years. This includes disciplinary programs in the various directorates and 
offices, and the cross-disciplinary programs managed across directorates. The COVs are 
comprised of scientists, engineers and educators who convene at NSE for a two to three day 
assessment. These experts evaluate the integrity and efficiency of the processes used for 
proposal review and program decision-making. In addition, the COVs provide a retrospective 
assessment of the quality of results of NSE’ s programmatic investments. The COV reports, 
written as answers and commentary to specific questions (see “Core Questions and Report 
Template,” Appendix Table 12 on pages 40-47) are submitted for review through Advisory 
Committees to the directorates and the NSE Director. Questions include aspects of the 
program portfolio, such as the balance of high-risk, multidisciplinary, and innovative 
projects. The recommendations of COVs are reviewed by management and taken into 
consideration by NSE when evaluating existing programs and future directions for the 
Eoundation. See Appendix Table 13, pages 48-57, for a schedule of COV program 
evaluations. 

• Advisory Committee (AC) Reporting on Directorate/Office Performance. Advisory 
committees regularly provide community perspectives to the research and education 
directorates, the Office of Polar Programs, and Office of International Science and 
Engineering. They are typically composed of 15-25 experts who have broad experience in 
academia, industry and government. The role of the ACs is to provide advice on priorities, 
address program effectiveness, review COV reports, and examine directorate/office 



The NSF Strategic Plan, FY 2003 - 2008, is available at 
<http://www.nsf.gOv/publications/pub_summ.j sp?ods_key=nsf0420 1 >. 

The COV reports and directorate responses are available electronically as a link from the NSF GPRA web page, 
<http://www.nsf. gov/ about/performance/> . 
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responses to COV recommendations. In FY 2001 and previous years, directorate/office 
advisory committees assessed directorate/office progress in achieving NSF-wide GPRA 
goals. With the advent of the AC/GPA (see below), advisory committees no longer assess 
directorate progress toward these goals. 

• Advisory Committee for GPRA Performance Assessment (AC/GPA) During FY 2002, 
NSF determined that a more efficient and effective process for the assessment of agency 
performance with respect to GPRA strategic goals was to charge a single external committee 
of experts with review of all Foundation accomplishments. The AC/GPA consists of 
approximately 25 external experts from various fields of science, engineering, mathematics 
and education. The AC/GPA looks at Foundation- wide portfolios linked to the agency’s 
strategic outcome goals of Ideas, People, Tools, and Organizational Excellence and their 
associated performance indicators. In June 2005, the AC/GPA convened to assess results, 
using COV reports, investigator project reports, and collections of outstanding 
accomplishments from awards as reported by NSF program officers. This external 
assessment found that, overall, in FY 2004, NSF achieved all four of its strategic outcome 
goals. With regard to merit review, the AC/GPA concluded "that the MRP [Merit Review 
Process] is effective in the processing and reviewing of a large and increasing volume of 
proposals and in the engagement of a broad and diverse segment of talent in the NSF's 
science and engineering enterprises. While the MRP will always, in our view, require 
vigilance and a commitment to continuous improvement, when taken as a whole and when 
one looks at the results as illustrated in the People, Ideas, and Tools portfolios, clearly the 
process remains a major positive force in advancing the frontiers of science, mathematics, 
and engineering.”^^ (See also discussion above, page 23.) 

• Assessment Utilizing the Program Assessment Rating Tool (PART). The Program 
Assessment Rating Tool was developed by the Office of Management and Budget (0MB) to 
assess program performance in four areas: Program Purpose and Design, Strategic Planning, 
Program Management, and Program Results / Accountability. In February 2005, results from 
PART assessments were released on the "Institutions," "Collaborations," and "Polar Research 
Tools, Facilities, and Logistics" programs and the Biocomplexity in the Environment priority 
area. All four areas were rated “effective,” the highest possible rating from 0MB for the 
PART. Again, NSE received the top three scores of all research and development programs 
assessed, and NSE programs were ranked with five in the top fifteen out of the over 600 
programs assessed across the entire government that year. Each year, additional programs 
will be assessed for the first time and previous assessments will be updated to reflect new 
information and actions taken to enhance program management and results. All NSE 
programs and current priority areas will be assessed by the end of EY 2008. 

• Independent Verification and Validation of Performance Measurement for the 
Government Performance and Results Act and the Program Assessment Rating Tool. 

NSE contracted with IBM Business Consulting Services to assess the validity of the data and 
reported results of NSE performance goals and to verify the reliability of the methods used 
by NSE to compile and report data for the performance measurement goals and objectives. 



Report of the Advisory Committee for GPRA Performance Assessment, July 2005, page 48. Available at 
http://www.nsf.gOv/pubs/2005/nsf052 1 0/nsf052 1 0.pdf. 



FY 2005 Report on the NSF Merit Review System 03/06 




29 



The contractor’s independent review, completed in October 2005, concluded that NSF made 
a concerted effort to report its performance results accurately and has effective systems, 
policies, and procedures to promote data quality. The review also verified that NSF relied on 
sound business policies and internal controls, and maintained adequate documentation of its 
processes and data.^^ 

Special Proposal and Grant Mechanisms 

Preliminary Proposals 

Some NSF programs invite the submission of preliminary proposals. The intent of preliminary 
proposals is to limit the burden imposed on proposers, reviewers and NSF staff. Normally, 
preliminary proposals require only enough information to make fair and reasonable decisions 
regarding encouragement/discouragement of a full proposal. Review practices for preliminary 
proposals range from non-binding advice from program officers to proposers to formal 
recommendations from external reviewers or panels. In FY 2005, NSF received a total of 
2,120 preliminary proposals, compared to 2,310 preliminary proposals in FY 2004 and 2,469 
preliminary proposals in FY 2003.^^ For those proposals subject to non-binding advice, NSF 
encouraged the submission of full proposals in 512 cases and discouraged submission of a full 
proposal in 790 cases. For the proposals subject to binding advice through formal 
recommendations, NSF invited the submission of a full proposal in 246 cases, and did not invite 
the submission of a full proposal in 570 cases. Two preliminary proposals were withdrawn. 

Small Grants for Exploratory Research (SGER) 

Since the beginning of FY 1990, the Small Grants for Exploratory Research (SGER) option has 
permitted program officers throughout the Eoundation to make small-scale grants without formal 
external review. Characteristics of activities that can be supported by an SGER award include: 
preliminary work on untested and novel ideas; ventures into emerging research and potentially 
transformative ideas; quick-response research on unanticipated events, such as natural disasters 
and infrequent phenomena; and similar efforts likely to catalyze rapid and innovative advances. 

Several science and engineering teams received SGERs to collect ephemeral data immediately 
following Hurricane Katrina. Program officers across the Eoundation welcomed proposals 
following the disaster and, through the SGER mechanism, were able to act swiftly to make the 
awards. 



IBM Business Consulting Services, “National Science Foundation: Government Performance and Results Act 
(GPRA) and Program Assessment Rating Tool (PART) Performance Measurement Validation and Verification, 
Report on FY 2005 Results,” October 2005. In NSF's FY 2005 Performance and Accountability Report, Section 2, 
page 92. Available on the web at <http://www.nsf.gov/about/performance/reports.jsp>. 

A binding (invite/non-invite) decision is the type of mechanism used when the NSF decision made on the 
preliminary proposal is final, affecting the Pi’s eligibility to submit a full proposal. A non-binding 
(encourage/discourage) decision is the type of mechanism used when the NSF decision made on the preliminary 
proposal is advisory only. This means that submitters of both favorably and unfavorably reviewed proposals are 
eligible to submit full proposals (Source: NSF Proposal and Award Manual). 

Please note that preliminary proposals are not included in the total count of proposals received and competitively 
reviewed at NSF as reported on page 5, above. 
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Potential SGER applicants are encouraged to contact an NSF program officer before submitting 
an SGER proposal to determine its appropriateness for funding. Directorate-level data on SGER 
proposals and awards are presented in Appendix Table 14, page 59. In EY 2005, NSE made 387 
SGER awards, compared to 382 awards in EY 2004, compared to and 344 awards in EY 2002. 
The total amount awarded to SGERs in EY 2005 was approximately $27 million compared to 
$29 million in the previous year. This represents about 0.5 percent of the operating budget for 
research and education. 

The average size of SGER award in EY 2005 was around $70,000, down from $77,000 in EY 
2004. In September 2003 NSE raised the maximum SGER award threshold from $100,000 to 
$200,000. Program officers may obligate no more than five percent of their program budget per 
fiscal year for SGER awards. NSE is initiating a study of the SGER portfolio in EY 2006. 

Accomplishment Based Renewals 

In an accomplishment-based renewal, the project description is replaced by copies of no more 
than six reprints of publications resulting from the research supported by NSE (or research 
supported by other sources that is closely related to the NSE-supported research) during the 
preceding three- to five-year period. In addition, a brief (not to exceed four pages) summary of 
plans for the proposed support period must be submitted. All other information required for NSE 
proposal submission remains the same. The proposals undergo merit review in the tradition of 
the specific program. In 2005 there were lOI requests for accomplishment-based renewals, 28 of 
which were awarded. 
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Appendix Table 1 

Competitively Reviewed Proposals, Awards and Success Rates 
By Directorate, FY 2001 - 2005 





Fiscal Year 




2001 


2002 


2003 


2004 


2005 


NSF 


Proposals 


31,942 


35,165 


40,075 


43,851 


41,722 




Awards 


9,925 


10,406 


10,844 


10,380 


9,757 




Funding Rate 


31% 


30% 


27% 


24% 


23% 


BIO 


Proposals 


5,131 


5,143 


5,591 


6,063 


6,475 




Awards 


1,431 


1,400 


1,448 


1,432 


1,355 




Funding Rate 


28% 


27% 


26% 


24% 


21% 


CISE 


Proposals 


3,578 


4,317 


5,270 


6,276 


5,238 




Awards 


884 


1,039 


1,175 


1,017 


1,088 




Funding Rate 


24% 


24% 


22% 


16% 


21% 


EHR 


Proposals 


3,449 


3,966 


4,111 


4,644 


3,699 




Awards 


1,157 


1,044 


890 


925 


736 




Funding Rate 


34% 


26% 


22% 


20% 


20% 


ENG 


Proposals 


5,983 


6,883 


9,076 


8,994 


8,692 




Awards 


1,426 


1,726 


1,945 


1,753 


1,493 




Funding Rate 


24% 


25% 


21% 


19% 


17% 


GEO 


Proposals 


3,580 


4,114 


4,230 


4,267 


4,676 




Awards 


1,417 


1,450 


1,515 


1,419 


1,315 




Funding Rate 


40% 


35% 


36% 


33% 


28% 


MRS 


Proposals 


5,692 


5,996 


6,694 


7,184 


7,083 




Awards 


1,996 


2,105 


2,268 


2,175 


2,071 




Funding Rate 


35% 


35% 


34% 


30% 


29% 


OCI 


Proposals 


288 


223 


342 


220 


116 




Awards 


39 


54 


56 


47 


75 




Funding Rate 


14% 


24% 


16% 


21% 


65% 


OPP 


Proposals 


634 


572 


557 


689 


816 




Awards 


201 


264 


241 


268 


281 




Funding Rate 


32% 


46% 


43% 


39% 


34% 


OISE 


Proposals 


610 


608 


670 


851 


822 




Awards 


358 


334 


373 


386 


333 




Funding Rate 


59% 


55% 


56% 


45% 


41% 


SBE 


Proposals 


2,900 


3,279 


3,491 


4,619 


4,089 




Awards 


942 


931 


894 


939 


1,004 




Funding Rate 


32% 


28% 


26% 


20% 


25% 


Other 


Proposals 


97 


64 


12 


44 


16 




Awards 


74 


59 


12 


19 


6 




Funding Rate 


76% 


92% 


100% 


43% 


38% 



Notes: 

• The following are not included in the above statistics: 8,307 CGIs, 3,616 

supplements, 420 contracts, and 2 cooperative agreements that underwent merit 
review in a prior FY. 



Source: NSF Enterprise Information System as of October 1, 2005. 
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Appendix Table 2 

Competitively Reviewed Proposals, Awards and Success Rates 
By PI Characteristics, FY 1998 - 2005 





1 Fiscal Year I 


1998 


1999 


2000 


2001 


2002 


2003 


2004 


2005 


All Pis 


Proposals 


28,422 


28,578 


29,508 


31,942 


35,165 


40,075 


43,851 


41,722 




Awards 


9,381 


9,189 


9,850 


9,925 


10,406 


10,844 


10,380 


9,757 




Funding Rate 


33% 


32% 


33% 


31% 


30% 


27% 


24% 


23% 


Female Pis 


Proposals 


5,627 


5,315 


5,509 


5,839 


6,704 




8,427 


8,266 




Awards 


1,938 


1,682 


1,949 


1,894 


2,012 


mm 


2,118 


2,107 




Funding Rate 


34% 


32% 


35% 


32% 


30% 


28% 


25% 


25% 


Male Pis 


Proposals 


22,513 


23,022 


23,671 


25,510 


27,500 


31,238 


33,300 


31 ,456 




Awards 


7,323 


7,428 


7,778 


7,867 


8,203 


8,495 


7,923 


7,305 




Funding Rate 


33% 


32% 


33% 


31% 


30% 


27% 


24% 


23% 


Minority Pis 


Proposals 


1,410 


1,434 


1,480 


1,728 


1,906 


2,141 


2,551 


2,468 




Awards 


403 


424 


472 


509 


548 


569 


597 


569 




Funding Rate 


29% 


30% 


32% 


29% 


29% 


27% 


23% 


23% 


New Pis 


Proposals 


12,255 


1 1 ,803 


12,327 


13,280 


15,085 


17,584 


19,052 


17,660 




Awards 


3,117 


2,689 


3,024 


3,136 


3,329 


3,390 


3,256 


3,001 




Funding Rate 


25% 


23% 


25% 


24% 


22% 


19% 


17% 


17% 


Prior Pis 


Proposals 


16,167 


16,775 


17,181 


18,662 


20,080 


22,511 


24,799 


24,062 




Awards 


6,264 


6,500 


6,826 


6,789 


7,077 


7,478 


7,124 


6,756 




Funding Rate 


39% 


39% 


40% 


36% 


35% 


33% 


29% 


28% 



Notes: 

• “Gender” is based on self-reported information from the Pi’s most reeent proposal. 

• “Minority” is based on the Pi’s ethnie/raeial status as reported to NSF on the most 
reeent proposal. 

• Pis ean decline to report their ethnic/racial status. Includes American Indian, Alaska 
Native, Black, Hispanic, and Pacific Islander and excludes Asian and White-Not of 
Hispanic Origin. 

Source: NSF Enterprise Information System as of October I, 2005. 
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Appendix Table 3 

Competitively Reviewed Proposals, Awards and Success Rates 
By Minority PI Ethnic/Racial Status, FY 1998 - 2005 





Fiscal Year 


1998 


1999 


2000 


2001 


2002 


2003 


2004 


2005 


American 


Proposals 


61 


58 


90 


118 


100 


112 


93 


94 


Indian/Alaska 


Awards 


17 


19 


34 


52 


30 


28 


23 


24 


Native 


Funding Rate 


28% 


33% 


38% 


44% 


30% 


25% 


25% 


26% 


Biack/ 


Proposals 


541 


539 


522 


668 


748 


822 


900 


813 


African 


Awards 


144 


146 


169 


180 


207 


192 


208 


193 


American 


Funding Rate 


27% 


27% 


32% 


27% 


28% 


23% 


23% 


24% 


Hispanic 


Proposals 


779 


807 


854 


955 


1,041 


1,191 


1,432 


1,436 


or 


Awards 


234 


245 


258 


285 


300 


342 


347 


322 


Latino 


Funding Rate 


30% 


30% 


30% 


30% 


29% 


29% 


24% 


22% 


Native 


Proposals 


46 


37 


41 


23 


32 


37 


47 


21 


Hawaiian/ 


Awards 


14 


13 


19 


6 


7 


12 


4 


4 


Pacific Island 


Funding Rate 


30% 


35% 


46% 


26% 


22% 


32% 


9% 


19% 



Source: NSF Enterprise Information System as of October 1, 2005. 



Appendix Table 4 

Median and Average Award Amounts by Directorate, 
Research Awards FY 2000 - 2005 





Fisca 


1 Year 




2000 


2001 


2002 


2003 


2004 


2005 


NSF 


Median 

Average 


$75,810 

$104,905 


$84,387 

$113,833 


$85,839 

$115,656 


$100,000 

$135,609 


$101,566 

$139,522 


$103,965 

$143,669 


BIO 


Median 

Average 


$99,854 

$117,378 


$108,333 

$143,512 


$110,000 

$136,509 


$126,000 

$177,305 


$133,191 

$171,074 


$140,000 

$183,939 


CISE 


Median 

Average 


$100,000 

$149,432 


$92,000 

$130,289 


$93,511 

$135,788 


$113,333 

$158,899 


$113,333 

$166,517 


$112,431 

$150,523 


ENG 


Median 

Average 


$75,000 

$87,601 


$80,946 

$99,506 


$83,965 

$102,060 


$99,997 

$119,470 


$96,677 

$119,704 


$97,054 

$117,456 


GEO 


Median 

Average 


$72,828 

$94,920 


$76,667 

$98,917 


$80,168 

$103,439 


$102,667 

$146,475 


$114,730 

$150,181 


$116,492 

$147,690 


MPS 


Median 

Average 


$75,100 

$108,804 


$86,243 

$114,421 


$83,319 

$111,617 


$100,000 

$128,585 


$100,000 

$130,043 


$100,000 

$135,423 


OCI 


Median 

Average 


$104,180 
$341 ,504 


$75,000 

$82,882 


$125,000 

$176,289 


$134,333 

$160,262 


$365,408 

$401,828 


$160,522 

$315,044 


OISE 


Median 

Average 


$7,939 

$14,193 


$8,784 

$17,429 


$9,800 

$16,441 


$10,000 

$20,869 


$10,000 

$15,003 


$14,996 

$90,980 


OPP 


Median 

Average 


$72,729 

$141,221 


$77,789 

$113,164 


$81,517 

$130,343 


$126,143 

$144,392 


$141,452 

$204,126 


$122,106 

$180,487 


SBE 


Median 

Average 


$52,778 

$60,538 


$63,377 

$80,709 


$62,950 

$78,035 


$77,388 

$89,488 


$77,948 

$90,373 


$84,050 

$110,184 



Source: NSF Enterprise Information System as of October I, 2005. 
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Appendix Table 5 
Methods of NSF Proposal Review 
FY 1993 - 2005 





Total 


Mall + Panel 


Mall-Only 


Panel-Only 


Not Externally 
Reviewed 


FY 


Proposals 


Proposals Percent 


Proposals Percent 


Proposals Percent 


Proposals Percent 


2005 


41,722 


13,919 


33% 


3,656 


9% 


22,735 


54% 


1,412 


3% 


2004 


43,851 


13,345 


31% 


4,496 


10% 


24,553 


56% 


1,457 


3% 


2003 


40,075 


12,683 


32% 


4,579 


11% 


21,391 


53% 


1,388 


3% 


2002 


35,164 


1 1 ,346 


32% 


4,838 


14% 


17,616 


50% 


1,364 


4% 


2001 


31,942 


9,367 


29% 


5,460 


17% 


15,751 


49% 


1,364 


4% 


2000 


29,507 


9,296 


32% 


6,048 


20% 


12,886 


44% 


1,277 


4% 


1999 


28,579 


8,918 


31% 


6,452 


23% 


12,046 


42% 


1,163 


4% 


1998 


28,422 


8,486 


30% 


6,974 


25% 


11,396 


40% 


1,566 


6% 


1997 


30,258 


8,812 


29% 


7,855 


26% 


12,109 


40% 


1,482 


5% 


1996 


30,199 


8,562 


28% 


7,812 


26% 


12,490 


41% 


1,335 


4% 


1995 


30,432 


8,400 


28% 


8,581 


28% 


11,912 


39% 


1,539 


5% 


1994 


30,336 


7,059 


23% 


8,687 


29% 


12,986 


43% 


1,604 


5% 


1993 


30,038 


7,032 


23% 


8,886 


30% 


12,338 


41% 


1,782 


6% 



Panel-Only includes cases where panel was mailed proposal for review prior to panel. 



Source: NSF Enterprise Information System as of October 1, 2005. 



Appendix Table 6 

Methods of NSF Proposal Review, By Directorates 
FY 2005 



Directorate 


Tetal 

Propesals 


Mail + Panel 


Mail-Only 


Panel-Only 


Net Externally 
Reviewed 


Propesals 


Percent 


Propesals Percent 


Proposals 


Percent 


Propesals Percent 


NSF 


41,722 


13,919 


33% 


3,656 


9% 


22,735 


54% 


1,412 


3% 


BIO 


6,475 


4,913 


76% 


56 


1% 


1,296 


20% 


210 


3% 


CISE 


5,238 


411 


8% 


95 


2% 


4,540 


87% 


192 


4% 


EHR 


3,699 


88 


2% 


95 


3% 


3,479 


94% 


37 


1% 


ENG 


8,692 


416 


5% 


210 


2% 


7,725 


89% 


341 


4% 


GEO 


4,676 


3,488 


75% 


585 


13% 


477 


10% 


126 


3% 


MPS 


7,083 


1,542 


22% 


1,872 


26% 


3,441 


49% 


228 


3% 


OCI 


116 


3 


3% 


14 


12% 


76 


66% 


23 


20% 


OISE 


822 


165 


20% 


332 


40% 


222 


27% 


103 


13% 


OPP 


816 


461 


56% 


249 


31% 


55 


7% 


51 


6% 


SBE 


4,089 


2,432 


59% 


139 


3% 


1,417 


35% 


101 


2% 


Other 


16 


0 


0% 


9 


56% 


7 


44% 


0 


0% 



Source: NSF Enterprise Information System as of October I, 2005. 
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Appendix Table 7 

Average Number of Reviews per Proposal 
By Method & Directorate, FY 2005 



1 


1 Methods of Review 




1 ^ 














Not 


Returned 
















Externally 


without 


Withdrawn 






All Methods 


Mail + Panel 


Mail-Only 


Panel-Only 


Reviewed * 


Review 


Proposals 


NSF 


Reviews 


246,273 


108,591 


15,552 


122,130 










Proposals 


40,310 


13,919 


3,656 


22,735 


1,412 


1,237 


398 




Rev/Prop 


6.1 


7.8 


4.3 


5.4 








BIO 


Reviews 


38,498 


32,807 


243 


5,448 










Proposals 


6,265 


4,913 


56 


1,296 


210 


288 


57 




Rev/Prop 


6.1 


6.7 


4.3 


4.2 








CSE 


Reviews 


26,470 


2,528 


360 


23,582 










Proposals 


5,046 


411 


95 


4,540 


192 


85 


39 




Rev/Prop 


5.2 


6.2 


3.8 


5.2 








EHR 


Reviews 


23,348 


493 


332 


22,523 










Proposals 


3,662 


88 


95 


3,479 


37 


106 


6 




Rev/Prop 


6.4 


5.6 


3.5 


6.5 








ENG 


Reviews 


42,066 


2,511 


854 


38,701 










Proposals 


8,351 


416 


210 


7,725 


341 


408 


49 




Rev/Prop 


5.0 


6.0 


4.1 


5.0 








GEO 


Reviews 


43,331 


37,284 


2,793 


3,254 










Proposals 


4,550 


3,488 


585 


477 


126 


30 


50 




Rev/Prop 


9.5 


10.7 


4.8 


6.8 








MRS 


Reviews 


40,634 


1 1 ,788 


8,200 


20,646 










Proposals 


6,855 


1,542 


1,872 


3,441 


228 


164 


120 




Rev/Prop 


5.9 


7.6 


4.4 


6.0 








OCI 


Reviews 


529 


19 


73 


437 










Proposals 


93 


3 


14 


76 


23 


0 


4 




Rev/Prop 


5.7 


6.3 


5.2 


5.8 








OISE 


Reviews 


3,206 


1,310 


1,073 


823 










Proposals 


719 


165 


332 


222 


103 


28 


29 




Rev/Prop 


4.5 


7.9 


3.2 


3.7 








OPP 


Reviews 


5,060 


3,695 


1,046 


319 










Proposals 


765 


461 


249 


55 


51 


4 


6 




Rev/Prop 


6.6 


8.0 


4.2 


5.8 








SBE 


Reviews 


23,040 


16,156 


546 


6,338 










Proposals 


3,988 


2,432 


139 


1,417 


101 


66 


38 




Rev/Prop 


5.8 


6.6 


3.9 


4.5 








Other 


Reviews 


91 


0 


32 


59 










Proposals 


16 


0 


9 


7 


0 


58 


0 




Rev/Prop 


5.7 


N/A 


3.6 


8.4 









Notes: 

• * The proposal totals shown in the "All Methods" category do not include the 
proposals shown in the "Not Externally Reviewed" category. Proposals which are not 
reviewed include SGERs and grants for travel and symposia. 

• The "Not Externally Reviewed" category includes award and decline actions such as 
SGERs and workshops, while the "Returned without Review" and "Withdrawn 
Proposal" categories reflect proposals which were neither awarded nor declined. 

• Panel reviews include panel summaries. There were 38,331 panel summaries in EY 
2005. 

• Peers participating as both a mail and a panel reviewer for the same proposal are 
counted as one review in this table. 

• Withdrawn proposals include only those that underwent merit review. 



Source: NSE Enterprise information System as of October 1, 2005. 
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Appendix Table 8 

Distribution of Average Reviewer Ratings 
Panel- Only Reviewed 



(0 

V> 

O 

a. 

o 



0 ) 

E 

3 




No Score PoortoFair FairtoGood Good to Very VeryGoodto Excellent 

Good Excellent 



□ Awards o Declines 



Note: 

• Number of FY 2005 Proposals - 18,454 Declines, 4,281 Awards 
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Appendix Table 9 

Distribution of Average Reviewer Ratings, 
Mail-Only Reviewed 



37 



1,200 




No Score PoortoFair FairtoGood Good to Very VeryGoodto Excellent 

Good Excellent 



□ Awards o Declines 



Note: 

• Number of FY 2005 Proposals - 2,435 Deelines, 1,221 Awards 
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Distribution of Average Reviewer Ratings 
Mail and Panel Reviewed 



38 
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2 12 



2,000 



0 



241 



7 



5,935 



2,194 




1,092 



433 



No Score PoortoFair FairtoGood Good to Very VeryGoodto Excellent 

Good Excellent 



□ Awards Q Declines 



Note: 

• Number of FY 2005 Proposals - 10,815 Declines, 3,104 Awards 
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Appendix Table 11 

Requests for Formal Reconsideration of Declined Proposals 
By Directorate, FY 2001-2005 





Fiscal Year 


2001 


2002 


2003 


2004 


2005 


First Level Reviews (by Assistant Directors): 


BIO Request 


8 


4 


4 


3 


2 


- Upheld 


6 


4 


4 


3 


2 


- Reversed 


2 


0 


0 


0 


0 


CISE Request 


1 


1 


1 


2 


3 


- Upheld 


1 


0 


0 


2 


3 


- Reversed 


0 


0 


1 


0 


0 


EHR Request 


4 


2 


3 


2 


7 


- Upheld 


3 


2 


3 


2 


7 


- Reversed 


1 


0 


0 


0 


0 


ENG Request 


1 


2 


2 


3 


3 


- Upheld 


1 


2 


2 


3 


3 


- Reversed 


0 


0 


0 


0 


0 


GEO Request 


2 


1 


4 


4 


0 


- Upheld 


2 


1 


4 


4 


0 


- Reversed 


0 


0 


0 


0 


0 


MRS Request 


24 


15 


4 


24 


15 


- Upheld 


22 


15 


4 


24 


15 


- Reversed 


2 


0 


0 


0 


0 


SBE Request 


2 


1 


2 


3 


3 


- Upheld 


1 


0 


2 


3 


3 


- Reversed 


1 


1 


1 


0 


0 


Other Request 


0 


0 


0 


0 


0 


- Upheld 


0 


0 


0 


0 


0 


- Reversed 


0 


0 


0 


0 


0 


Second Level Reviews (by Deputy Director): 


O/DD Request 


2 


4 


5 


7 


2 


- Upheld 


1 


4 


4 


7 


2 


- Reversed 


0 


0 


1 


0 


0 


Total Reviews First & Second Level 


NSF Request 


44 


30 


26 


49 


35 


- Upheld 


37 


29 


24 


48 


35 


- Reversed 


6 


1 


2 


1 


0 



Note: 

The number of decisions (upheld or reversed) may not equal the 
number of requests in each year due to the carryover of the 
pending reconsideration request. 

Source: Office of the Director 
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Appendix Table 12 

CORE QUESTIONS and REPORT TEMPLATE 

for 

EY 2005 NSE COMMITTEE OE VISITOR (COV) REVIEWS 

Guidance to NSE Staff: This document includes the FY 2005 set of Core Questions and the 
COV Report Template for use hy NSF staff when preparing and conducting COVs during FY 
2005. Specific guidance for NSF staff describing the COV review process is described in 
Subchapter 300-Committee of Visitors Reviews (NSF Manual 1, Section VIII) that can be 
obtained at http ://w w w . inside. nsf. gov/od/gpra/. 

NSF relies on the judgment of external experts to maintain high standards of program 
management, to provide advice for continuous improvement of NSF performance, and to ensure 
openness to the research and education community served by the Foundation. Committee of 
Visitor (COV) reviews provide NSF with external expert judgments in two areas: (1) assessments 
of the quality and integrity of program operations and program-level technical and managerial 
matters pertaining to proposal decisions; and (2) comments on how the results generated by 
awardees have contributed to the attainment of NSF’ s mission and strategic outcome goals. 

Many of the Core Questions are derived from NSF performance goals and apply to the portfolio 
of activities represented in the program(s) under review. The program(s) under review may 
include several subactivities as well as NSF-wide activities. The directorate or division may 
instruct the COV to provide answers addressing a cluster or group of programs - a portfolio of 
activities integrated as a whole - or to provide answers specific to the subactivities of the 
program, with the latter requiring more time but providing more detailed information. 

The Division or Directorate may choose to add questions relevant to the activities under review. 
NSF staff should work with the COV members in advance of the meeting to provide them with 
the report template, organized background materials, and to identify questions/goals that apply to 
the program(s) under review. 

Guidance to the COV: The COV report should provide a balanced assessment of NSF’s 

performance in two primary areas: (A) the integrity and efficiency of the processes related to 
proposal review; and (B) the quality of the results of NSF’s investments that appear over time. 
The COV also explores the relationships between award decisions and program/NSF-wide goals 
in order to determine the likelihood that the portfolio will lead to the desired results in the future. 
Discussions leading to answers for Part A of the Core Questions will require study of confidential 
material such as declined proposals and reviewer comments. COV reports should not contain 
confidential material or specific information about declined proposals. Discussions leading to 
answers for Part B of the Core Questions will involve study of non-confidential material such as 
results of NSF-funded projects. It is important to recognize that the reports generated by COVs 
are used in assessing agency progress in order to meet government-wide performance reporting 
requirements, and are made available to the public. Since material from COV reports is used in 
NSF performance reports, the COV report may be subject to an audit. 

We encourage COV members to provide comments to NSF on how to improve in all areas, as 
well as suggestions for the COV process, format, and questions. 
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FY 2005 REPORT TEMPLATE EOR 
NSE COMMITTEES OE VISITORS (COVs) 



DateofCOV 

Program/Cluster: 

Division: 

Directorate: 

Number of actions reviewed by COV^**: Awards: Declinations: Other: 

Total number of actions within Program/Cluster/Division during period being reviewed by COV^^: 

Awards: Declinations: Other: 

Manner in which reviewed actions were selected: 



PART A. INTEGRITY AND EFFICIENCY OF THE PROGRAM’S PROCESSES 
AND MANAGEMENT 

Briefly discuss and provide comments for each relevant aspect of the program's review process and 
management. Comments should he based on a review of proposal actions (awards, declinations, 
and withdrawals) that were completed within the past three fiscal years. Provide comments for 
each program being reviewed and for those questions that are relevant to the program under review. 
Quantitative information may be required for some questions. Constructive comments noting areas 
in need of improvement are encouraged. 

A.l Questions about the quality and effectiveness of the program’s use of merit review 
procedures. Provide comments in the space below the question. Discuss areas of concern in the 
space provided. 



QUALITY AND EFFECTIVENESS OF MERIT REVIEW PROCEDURES 


YES, NO, 
DATA NOT 
AVAILABLE, 
or NOT 

APPLICABLE^* 


Is the review mechanism appropriate? (panels, ad hoc reviews, site visits) 
Comments: 




Is the review process efficient and effective? 
Comments: 





To be provided by NSF staff. 

To be provided by NSF staff. 

If “Not Applicable” please explain why in the “Comments” section. 
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Are reviews consistent with priorities and criteria stated in the program’s solicitations, 
announcements, and guidelines? 

Comments: 




Do the individual reviews (either mail or panel) provide sufficient information for the 
principal investigator(s) to understand the basis for the reviewer’s recommendation? 
Comments: 




Do the panel summaries provide sufficient information for the principal investigator(s) to 
understand the basis for the panel recommendation? 

Comments: 




Is the documentation for recommendations complete, and does the program officer 
provide sufficient information and justification for her/his recommendation? 
Comments: 




Is the time to decision appropriate? 
Comments: 




Discuss issues identified by the COV concerning the quality and effectiveness of the program’s use of 
merit review procedures: 



A.2 Questions concerning the implementation of the NSF Merit Review Criteria 
(intellectual merit and broader impacts) hy reviewers and program officers. Provide 
comments in the space below the question. Discuss issues or concerns in the space provided. 



IMPLEMENTATION OE NSE MERIT REVIEW CRITERIA 


YES, NO, 
DATA NOT 
AVAILABLE, or 
NOT 

APPLICABLE^’ 


Have the individual reviews (either mail or panel) addressed whether the proposal 
contributes to both merit review criteria? 

Comments: 





In “Not Applicable” please explain why in the “Comments” section. 
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Have the panel summaries addressed both merit review criteria? 
Comments: 




Have the review analyses (Form 7s) addressed both merit review criteria? 
Comments: 




Discuss any issues the COV has identified with respect to implementation of NSF’ s merit review 
criteria. 



A.3 Questions concerning the selection of reviewers. Provide comments in the space below the 
question. Discuss areas of concern in the space provided. 



SELECTION OE REVIEWERS 


YES , NO, 
DATA NOT 
AVAILABLE, 
or NOT 

APPLICABLE^® 


Did the program make use of an adequate number of reviewers? 
Comments: 




Did the program make use of reviewers having appropriate expertise and/or 
qualifications? 

Comments: 




Did the program make appropriate use of reviewers to reflect balance among 
characteristics such as geography, type of institution, and underrepresented groups? 
Comments: 




Did the program recognize and resolve conflicts of interest when appropriate? 
Comments: 




Discuss any issues the COV has identified relevant to selection of reviewers. 



A.4 Questions concerning the resulting portfolio of awards under review. Provide 
comments in the space below the question. Discuss areas of concern in the space provided. 



If “Not Applicable” please explain why in the “Comments” section. 
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RESULTING PORTFOLIO OF AWARDS 


APPROPRIATE, 
NOT APPROPRIATE^^ 
OR DATA NOT 
AVAILABLE 


Overall quality of the research and/or education projects supported hy the program. 
Comments: 




Are awards appropriate in size and duration for the scope of the projects? 
Comments: 




Does the program portfolio have an appropriate balance of: 
• High risk projects? 

Comments: 




Does the program portfolio have an appropriate balance of: 
• Multidisciplinary projects? 

Comments: 




Does the program portfolio have an appropriate balance of: 
• Innovative projects? 

Comments: 




Does the program portfolio have an appropriate balance of: 

• Funding for centers, groups and awards to individuals? 
Comments: 




Does the program portfolio have an appropriate balance of: 
• Awards to new investigators? 

Comments: 




Does the program portfolio have an appropriate balance of: 

• Geographical distribution of Principal Investigators? 
Comments: 




Does the program portfolio have an appropriate balance of: 
• Institutional types? 

Comments: 





If “Not Appropriate” please explain why in the “Comments” section. 
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Does the program portfolio have an appropriate balance of: 
• Projects that integrate research and education? 
Comments: 




Does the program portfolio have an appropriate balance: 

» Across disciplines and subdisciplines of the activity and of emerging 
opportunities? 

Comments: 




Does the program portfolio have appropriate participation of underrepresented 
groups? 

Comments: 




Is the program relevant to national priorities, agency mission, relevant fields and 
other customer needs? Include citations of relevant external reports. 

Comments: 




Discuss any concerns identified that are relevant to the quality of the projects or the balance of the portfolio. 



A.5 Management of the program under review. Please comment on: 



Management of the program. 
Comments: 



Responsiveness of the program to emerging research and education trends. 
Comments: 



Program planning and prioritization process (internal and external) that guided the development of the 
portfolio under review. 

Comments: 



Additional concerns relevant to the management of the program. 
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PART B. RESULTS OF NSF INVESTMENTS 

NSF investments produce results that appear over time. The answers to the first three (People, 
Ideas and Tools) questions in this section are to he based on the COV’s study of award results, 
which are direct and indirect accomplishments of projects supported hy the program. These 
projects may he currently active or closed out during the previous three fiscal years. The COV 
review may also include consideration of significant impacts and advances that have developed 
since the previous COV review and are demonstrably linked to NSF investments, regardless of 
when the investments were made. Incremental progress made on results reported in prior fiscal 
years may also be considered. 

The following questions are developed using the NSF outcome goals in the NSF Strategic Plan. 
The COV should look carefully at and comment on (1) noteworthy achievements of the year 
based on NSF awards; (2) the ways in which funded projects have collectively affected progress 
toward NSF’s mission and strategic outcomes; and (3) expectations for future performance based 
on the current set of awards. NSF asks the COV to provide comments on the degree to which past 
investments in research and education have contributed to NSF’s progress towards its annual 
strategic outcome goals and to its mission: 

• To promote the progress of science. 

• To advance national health, prosperity, and welfare. 

• To secure the national defense. 

• And for other purposes. 

Excellence in managing NSF underpins all of the agency’s activities. For the response to the 
Outcome Goal for Organizational Excellence, the COV should comment, where appropriate, on 
NSE providing an agile, innovative organization. Critical indicators in this area include (1) 
operation of a credible, efficient merit review system; (2) utilizing and sustaining broad access to 
new and emerging technologies for business application; (3) developing a diverse, capable, 
motivated staff that operates with efficiency and integrity; and (4) developing and using 
performance assessment tools and measures to provide an environment of continuous 
improvement in NSE’s intellectual investments as well as its management effectiveness. 

B. Please provide comments on the activity as it relates to NSF’s Strategic Outcome Goals. 
Provide examples of outcomes (nuggets) as appropriate. Examples should reference the 
NSF award number, the Principal Investigator(s) names, and their institutions. 



B.l OUTCOME GOAL for PEOPLE : Developing “a diverse, competitive and globally engaged 
workforce of scientists, engineers, technologists and well-prepared citizens.” 

Comments: 

B.2 OUTCOME GOAL for IDEAS : Enabling “discovery across the frontier of science and engineering, 
connected to learning, innovation, and service to society.” 

Comments: 
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B.3 OUTCOME GOAL for TOOLS: Providing “broadly accessible, state-of-the-art S&E 
facilities, tools and other infrastructure that enable discovery, learning and innovation.” 

Comments: 

B.4 OUTCOME GOAL for ORGANIZATIONAL EXCELLENCE : Providing “an agile, innovative 
organization that fulfills its mission through leadership in state-of-the-art business practices.” 

Comments: 



PARTC. OTHER TOPICS 

C.l Please comment on any program areas in need of improvement or gaps (if any) within 
program areas. 

C.2 Please provide comments as appropriate on the program’s performance in meeting 
program-specific goals and objectives that are not covered by the above questions. 

C.3 Please identify agency-wide issues that should be addressed by NSE to help improve the 
program's performance. 

C.4 Please provide comments on any other issues the COV feels are relevant. 

C.5 NSE would appreciate your comments on how to improve the COV review process, 
format and report template. 

SIGNATURE BLOCK: 



For the [Replace with Name of COV] 
[Name of Chair of COV] 

Chair 
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Committee of Visitors Meetings 
By Directorate 



(COV meetings held during FY 2005 are highlighted in bold^ 


» 


DIRECTORATE 


Fiscal 


Fiscal 


Division 


Year of 


Year of 


Program or Cluster 


Most 


Next 




Recent 


COV 




COV 




BIOLOGICAL SCIENCES 






Biological Infrastructure 


2004 


2007 


Research Resources (includes former Instrument-Related Activities) 


2004 


2007 


Human Resources (includes former Training Cluster) 


2004 


2007 


Plant Genome Research Program 


2004 


2007 


Environmental Biology 


2003 


2006 


Ecological Biology (Ecol. Studies held COV in 2002) 


2002 


2006 


Ecosystem Science (Thematic Review held COV in 2001) 


2001 


2006 


Population and Evolutionary Processes (Systematic and Population Biology 






held COV in 2000) 


2000 


2006 


Systematic Biology and Biodiversity Inventories 




2006 


Integrative Organismal Biology (formerly Int. Biology and Neuroscience) 


2005 


2008 


Behavioral Systems 


2005 


2008 


Developmental Systems 


2005 


2008 


Environmental and Structural Systems 


2005 


2008 


Functional and Regulatory Systems 


2005 


2008 


Molecular and Cellular Biosciences 


2005 


2008 


Biomolecular Systems (formerly Biomolecular Structure and Function 


2005 


2008 


and Biomolecular Processes) 


2005 


2008 


Cellular Systems (formerly Cell Biology) 


2005 


2008 


Genes and Genome Systems (formerly Genetics) 


2005 


2008 


Emerging Erontiers ( new in ’03 ) 


N/A 


2006 
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COMPUTER AND INFORMATION SCIENCE AND ENGINEERING 






Please note that CISE programs and divisions were reorganized in FY 2003. COVs 






for IIS, ANIR, and CCR were held in FY 2003. 






Computing & Communication Foundations ( CCF) 




2006 


Emerging Models & Technologies for Computation 




2006 


Formal & Mathematical Foundations 




2006 


Foundations of Computing Processes & Artifacts 




2006 


Computer & Network Systems (CNS) 




2006 


Computer Systems 




2006 


Computing Research Infrastructure 




2006 


Education & Workforce 




2006 


Network Systems 




2006 


Information & Intelligent Systems (IIS) 




2006 


Data, Inference & Understanding 




2006 


Science & Engineering Informatics 




2006 


Systems in Context 






Shared Cyberinfrastructure (SCI) 


2005 


2008 
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EDUCATION AND HUMAN RESOURCES 






Educational Systemic Reform (discontinued) 






Statewide Systemic Initiatives 


2004 




Urban Systemic Initiatives 


2004 




Rural Systemic Initiatives 


2004 




Office of Innovation Partnerships 






EPSCoR 


2005 


2008 


EDUCATION AND HUMAN RESOURCES (continued) 






Elementary, Secondary and Informal Education 






Informal Science Education 


2005 


2008 


Teacher Enhancement (now Teacher Professional Continuum) 


2003 


2006 


Instructional Materials Development 


2005 


2008 


Centers for Learning and Teaching (new in ’01) 


2004 


2007 


Undergraduate Education 






Teacher Preparation (subsumed under Teacher Professional Continuum) 






Advanced Technological Education 


2003 


2006 


NSE Computer, Science, Engineering and Mathematics 






Scholarships (new in ’01) 


2003 


2006 


Distinguished Teaching Scholars (new in ’02) 


2005 


2008 


Scholarship for Service (new in ’01) 


2004 


2007 


National SMETE Digital Library (new in ’01) 


2005 


2008 


Course, Curriculum, and Laboratory Improvement 


2003 


2006 


The STEM Talent Expansion Program (STEP) (new in ’02) 


2006 


2009 


Graduate Education 






Graduate Research Eellowships 


2003 


2006 


NATO Post doctorate Eellowships (program discontinued) 


2004 




IGERT (new in ’97) 


2005 


2008 


GK-12 Fellows (new in ’99) 


2005 


2008 


Human Resource Development 






The Louis Stokes Alliances for Minority Participation 


2005 


2008 


Centers for Research Excellence in Science and Technology (CREST) 


2005 


2008 


Programs for Gender Equity (PGE) 


2003 


2006 


Programs for Persons with Disabilities (PPD) 


2003 


2006 


Alliances for Graduate Education and the Professoriate (AGEP) 


2005 


2008 


Tribal Colleges Program (TCP) (new in ’01) 


2004 


2007 


Historically Black Colleges and Universities (HBCU) 


2005 


2008 
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Research, Evaluation & Communications 






REPP/ROLE (new in ’96) 


2005 


2008 


Evaluation 


2003 


2006 


Interagency Education Research Initiative (lERI) (new in ’01) 


2005 


2008 


Other 






H-IB VISA K-12 


2005 




Math and Science Partnership (MSP) (new in ’02) 


2005 





ENGINEERING 






Bioengineering and Environmental Systems 


2005 


2008 


Biochemical Engineering & Technology 


2005 


2008 


Biomedical Engineering & Research to Aid Persons with Disabilities 


2005 


2008 


Environmental Engineering & Technology 


2005 


2008 


Civil and Mechanical Systems 


2004 


2007 


Dynamic System Modeling, Sensing and Control 


2004 


2007 


Geotechnical and GeoHazard Systems 


2004 


2007 


Infrastructure and Information Systems 


2004 


2007 


Solid Mechanics and Materials Engineering 


2004 


2007 


Structural Systems and Engineering 


2004 


2007 


Network for Earthquake Engineering Simulation 


2004 


2007 


Chemical and Transport Systems 




2006 


Chemical Reaction Processes 


2003 


2006 


Interfacial, Transport and Separation Processes 


2003 


2006 


Eluid and Particle Processes 


2003 


2006 


Thermal Systems 


2003 


2006 


Design, Manufacture and Industrial Innovation 






-Engineering Decision Systems Programs (new in ’02) 


2003 


2006 


Engineering Design 


2003 


2006 


Manufacturing Enterprise Systems (new in ’02) 


2003 


2006 


Service Enterprise Systems (new in ’02) 


2003 


2006 


Operations Research 


2003 


2006 


-Manufacturing Processes and Equipment Systems 


2003 


2006 


Materials Processing and Manufacturing 


2003 


2006 


Manufacturing Machines and Equipment 


2003 


2006 


Nanomanufacturing (new in ’02) 


2003 


2006 
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-Small Business 






Small Business Innovation Research (SBIR) 


2004 


2007 


Small Business Technology Transfer 


2004 


2007 


-Crosscutting 






Grant Opportunities for Academic Liaison w/ Industry 


2003 


2006 


Innovation and Organizational Change 


2003 


2006 


Electrical and Communications Systems 






Electronics, Photonics and Device Technologies 


2005 


2008 


Control, Networks, and Computational Intelligence 


2005 


2008 


Integrative Systems (new in ’02) 


2005 


2008 


Engineering, Education and Centers 






Engineering Education 


2004 


2007 


Engineering Research Centers 


2004 


2007 


Industry /University Cooperative Research Centers 


2004 


2007 


Partnerships for Innovation (new in ’01) 


2004 


2007 
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GEOSCIENCES 






Atmospheric Sciences 






-Lower Atmosphere Research Section 






Atmospheric Chemistry 


2004 


2007 


Climate Dynamics 


2004 


2007 


Mesoscale Dynamic Meteorology 


2004 


2007 


Large-scale Dynamic Meteorology 


2004 


2007 


Physical Meteorology 


2004 


2007 


Paleoclimate 


2004 


2007 


-Upper Atmosphere Research Section 






Magnetospheric Physics 


2005 


2008 


Aeronomy 


2005 


2008 


Upper Atmospheric Research Facilities 


2005 


2008 


Solar Terrestrial Research 


2005 


2008 


-UCAR and Lower Atmospheric Facilities Oversight Section 






Lower Atmospheric Observing Facilities 


2003 


2006 


UNIDATA 


2003 


2006 


NCAR/UCAR 


2003 


2006 


Earth Sciences 






Instrumentation and Facilities 


2004 


2007 


-Research Support 






Tectonics 


2005 


2008 


Geology and Paleontology 


2005 


2008 


Hydrological Sciences 


2005 


2008 


Petrology and Geochemistry 


2005 


2008 


Geophysics 


2005 


2008 


Continental Dynamics 


2005 


2008 


Ocean Sciences 






-Integrative Programs Section 






Oceanographic Technical Services 


2005 


2008 


Ship Operations 


2005 


2008 


Oceanographic Instrumentation 


2005 


2008 


Ship Acquisitions and Upgrades (new in ’02) 


2005 


2008 


Shipboard Scientific Support Equipment (new in ’02) 


2005 


2008 


Oceanographic Tech and Interdisciplinary Coordination 


2003 


2006 


Ocean Science Education and Human Resources 


2003 


2006 


-Marine Geosciences Section 






Marine Geology and Geophysics 


2003 


2006 


Ocean Drilling 


2003 


2006 
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-Ocean Section 






Chemical Oceanography 


2003 


2006 


Physical Oceanography 


2003 


2006 


Biological Oceanography 


2003 


2006 


Other Programs 






Global Learning and Observation to Benefit the Environment 


2003 


2006 


Opportunities to Enhance Diversity in the Geosciences 


2003 


2006 


Geoscience Education 


2003 


2006 



MATHEMATICAL AND PHYSICAL SCIENCES 






Astronomical Sciences 


2005 


2008 


Planetary Astronomy 


2005 


2008 


Stellar Astronomy and Astrophysics 


2005 


2008 


Galactic Astronomy 


2005 


2008 


Education, Human Resources and Special Programs 


2005 


2008 


Advanced Technologies and Instrumentation 


2005 


2008 


Electromagnetic Spectrum Management 


2005 


2008 


Extragalactic Astronomy and Cosmology 


2005 


2008 


-Facilities Cluster 






Gemini Observatory 


2005 


2008 


National Radio Astronomy Observatory (NRAO) 


2005 


2008 


National Optical Astronomy Observatories (NOAO) 


2005 


2008 


National Solar Observatory (NSO) 


2005 


2008 


National Astronomy and Ionosphere Center (NAIC) 


2005 


2008 


Atacama Large Millimeter Array (ALMA) 


2005 


2008 


Chemistry 


2004 


2007 


Analytical & Surface Chemistry 


2004 


2007 


Chemistry Research Instrumentation and Eacilities 


2004 


2007 


Collaborative Research in Chemistry 


2004 


2007 


Inorganic, Bioinorganic and Organometallic Chemistry 


2004 


2007 


Organic & Macromolecular Chemistry 


2004 


2007 


Physical Chemistry 


2004 


2007 


Undergraduate Research Centers (pilot program, new in ‘04) 




2007 


Materials Research 


2005 


2008 


-Base Science Cluster 






Condensed Matter Physics 


2005 


2008 


Solid-State Chemistry 


2005 


2008 


Polymers 


2005 


2008 
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-Advanced Materials and Processing Cluster 






Metals 


2005 


2008 


Ceramics 


2005 


2008 


Electronic Materials 


2005 


2008 


-Materials Research and Technology Enabling Cluster 






Materials Theory 


2005 


2008 


Instrumentation for Materials Research 


2005 


2008 


National Facilities 


2005 


2008 


Materials Research Science and Engineering Centers 


2005 


2008 


-Office of Special Programs (new in ’03) 


N/A 


2008 


Mathematical Sciences 


2004 


2007 


Applied Mathematics 


2004 


2007 


Geometric Analysis, Topology and Foundations 


2004 


2007 


Computational Mathematics 


2004 


2007 


Infrastructure 


2004 


2007 


Analysis 


2004 


2007 


Algebra, Number Theory, and Combinatorics 


2004 


2007 


Statistics and Probability 


2004 


2007 


Mathematical Biology (new in ‘04) 




2007 


Physics 






Atomic, Molecular, Optical and Plasma Physics 


2003 


2006 


Elementary Particle Physics 


2003 


2006 


Theoretical Physics 


2003 


2006 


Particle and Nuclear Astrophysics (new in ’00) 


2003 


2006 


Nuclear Physics 


2003 


2006 


Biological Physics (new in ’03) 




2006 


Physics at the Information Frontier (new in ’03) 




2006 


Physics Frontier Centers (new in ’02) 




2006 


Education and Interdisciplinary Research (new in ’00) 


2003 


2006 


Gravitational Physics 


2003 


2006 


Office of MultidisciplinaryResearch 


2003 


2006 
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SOCIAL, BEHAVIORAL, AND ECONOMIC SCIENCES 






Science Resource Statistics (SRS) 






All programs 




2006 


Behavioral and Cognitive Sciences (BCS) 






Cultural Anthropology 


2003 


2006 


Linguistics 


2003 


2006 


Social Psychology 


2003 


2006 


Physical Anthropology 


2003 


2006 


Geography and Regional Sciences 


2003 


2006 


Cognitive Neuroscience (new in ’01) 


2003 


2006 


Developmental and Learning Sciences (formally Child Learning & 


2003 


2006 


Development) 






Perception, Action, and Cognition (formally Human Cognition & 


2003 


2006 


Perception) 






Archaeology 


2003 


2006 


Archaeometry (formally part of Archaeology) 


2003 


2006 


Environmental Social and Behavioral Science (new in ’99) 


2003 


2006 


Social and Economic Sciences (SES) 






Decision, Risk, and Management Sciences 


2004 


2007 


Political Science 


2004 


2007 


Law and Social Science 


2004 


2007 


Innovation and Organizational Change 


2004 


2007 


Methodology, Measurement and Statistics 


2004 


2007 


Science and Technology Studies 


2004 


2007 


Societal Dimensions of Engineering, Science, and Technology 


2004 


2007 


Economics 


2004 


2007 


Sociology 


2004 


2007 


ADVANCE (Cross-Directorate Program, new in FY01/FY02) 


2005 


2008 


Science of Learning Centers (new in FY03/FY04) 




2007 


Human and Social Dynamics ( new in FY04 ) 




2008 
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OFFICE OF POLAR PROGRAMS 






Polar Research Support 


2004 


2007 


Antarctic Sciences 


2003 


2006 


Antarctic Aeronomy and Astrophysics 


2003 


2006 


Antarctic Biology and Medicine 


2003 


2006 


Antarctic Geology and Geophysics 


2003 


2006 


Antarctic Glaciology 


2003 


2006 


Antarctic Ocean and Climate Systems 


2003 


2006 


Arctic Sciences 






Arctic Research Support and Logistics 


2003 


2006 


Arctic System Sciences 


2003 


2006 


Arctic Natural Sciences 


2003 


2006 


Arctic Social Sciences 


2003 


2006 



OFFICE OF INTEGRATIVE ACTIVITIES 






Major Research Instrumentation (MRI) 


2005 


2008 


Science and Technology Centers (STC) 


1996* 


2007* 


*External Evaluations 







OFFICE OF INTERNATIONAL SCIENCE & ENGINEERING 



2005 



2008 



NSF PRIORITY AREAS AND CROSSCUTTING PROGRAMS 






Nanoscale Science and Engineering Priority Area 


2004 


2007 


Biocompexity in the Environment 


2004 


2007 


CAREER 


2001 


2006* 


Information Technology Research (new in ’00; no longer active) 


2005 




*External Evaluations 
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Small Grants for Exploratory Research (SGER) 
Funding Trends by Directorate, FY 2003 - 2005 





Fiscal Year | 


2003 


2004 


2005 


NSF Proposals 

Awards 
Total $ 

% of Obligations 
Average $ 


435 

344 

$23,424,191 

0.4% 

$68,094 


640 

382 

$29,493,932 

0.5% 

$77,209 


504 

387 

$26,980,122 

0.5% 

$69,716 


BIO Proposals 

Awards 
Total $ 

% of Obligations 
Average $ 


52 

48 

$3,417,138 

0.6% 

$71,190 


65 

52 

$5,392,558 

0.9% 

$103,703 


55 

38 

$3,020,321 

0.5% 

$79,482 


CISE Proposals 

Awards 
Total $ 

% of Obligations 
Average $ 


59 

51 

$3,934,783 

0.8% 

$78,133 


51 

48 

$3,170,389 

0.6% 

$87,814 


82 

71 

$6,678,905 
1 .4% 
$94,069 


EHR Proposals 

Awards 
Total $ 

% of Obligations 
Average $ 


6 

5 

$418,335 

0.1% 

$83,667 


17 

16 

$2,092,916 

0.2% 

$130,807 


15 

11 

$1,498,645 

0.2% 

$136,240 


ENG Proposals 

Awards 
Total $ 

% of Obligations 
Average $ 


128 

110 

$7,522,161 

1.3% 

$68,383 


127 

119 

$8,147,351 
1 .4% 
$68,465 


176 

126 

$6,708,778 

1.1% 

$53,244 


GEO Proposals 

Awards 
Total $ 

% of Obligations 
Average $ 


62 

60 

$2,915,587 

0.4% 

$48,593 


68 

64 

$3,508,457 

0.4% 

$54,820 


62 

59 

$3,414,557 

0.5% 

$57,874 


MPS Proposals 

Awards 
Total $ 

% of Obligations 
Average $ 


97 

43 

$3,820,670 

0.3% 

$88,853 


272 

45 

$4,423,294 

0.4% 

$98,295 


21 

18 

$1,663,544 

0.1% 

$92,419 


OCI Proposals 

Awards 
Total $ 

% of Obligations 
Average $ 


0 

0 

$50,000 

0.0% 

N/A 


0 

0 

$1,044,683 

0.8% 

N/A 


11 

11 

$1,458,472 
1 .2% 
$132,588 


OISE Proposals 

Awards 
Total $ 

% of Obligations 
Average $ 


0 

0 

$59,326 

0.1% 

N/A 


0 

0 

$62,200 

0.2% 

N/A 


0 

0 

$102,000 

0.2% 

N/A 


OPP Proposals 

Awards 
Total $ 

% of Obligations 
Average $ 


14 

13 

$681,087 

0.2% 

$52,391 


18 

16 

$695,961 

0.2% 

$43,498 


24 

24 

$1,197,306 

0.3% 

$49,888 


SBE Proposals 

Awards 
Total $ 

% of Obligations 
Average $ 


17 

14 

$605,104 

0.4% 

$47,459 


22 

22 

$820,999 

0.4% 

$37,318 


58 

29 

$1,237,594 

0.6% 

$42,676 
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Terms & Acronyms 



Acronym Definition 



A&M 

AC 

AD 

BFA 

BIO 

CAREER 

CGI 

CISE 

COV 

EHR 

EIS 

ENG 

EPSCoR 

EERDC 

ETE 

EY 

GPRA 

lA 

IPA 

IPS 

IPERS 

MPS 

NSE 

OCI 

ODS 

OIG 

OISE 

0MB 

OPP 

PARS 

PI 

R&D 

R&RA 

S&E 

S&E 

SBE 

SGER 

VSEE 



Administration and Management 
Advisory Committee 
Assistant Director 

Office of Budget, Einance and Award Management 
Directorate for Biological Sciences 
Eaculty Early Career Development Program 
Continuing Grant Increments 

Directorate for Computer and Information Science and Engineering 
Committee of Visitors 

Directorate for Education and Human Resources 
Enterprise Information System 
Directorate for Engineering 

Experimental Program to Stimulate Competitive Research 
Eederally Eunded Research and Development Center 
Eull-Time Equivalent 
Eiscal Year 

Government Performance and Results Act 
Integrative Activities 

Intergovernmental Personnel Act (appointee) 

Interactive Panel System 

Integrated Personnel System 

Directorate for Mathematical and Physical Sciences 

National Science Eoundation 

Office of Cyberinfrastructure 

Online Document System 

Office of Inspector General 

Office of International Science & Engineering 

Office of Management and Budget 

Office of Polar Programs 

Proposal, PI and Reviewer System 

Principal Investigator 

Research and Development 

Research and Related Activities (account) 

Science and Engineering 
Salaries and Expenses (account) 

Directorate for Social, Behavioral and Economic Sciences 
Small Grant for Exploratory Research 
Visiting Scientists, Engineers and Educators 



FY 2005 Report on the NSF Merit Review System — 03/06 




